2020
pdf
abs
Dynamic Topic Tracker for KBtoText Generation
Zihao Fu

Lidong Bing

Wai Lam

Shoaib Jameel
Proceedings of the 28th International Conference on Computational Linguistics
Recently, many KBtotext generation tasks have been proposed to bridge the gap between knowledge bases and natural language by directly converting a group of knowledge base triples into humanreadable sentences. However, most of the existing models suffer from the offtopic problem, namely, the models are prone to generate some unrelated clauses that are somehow involved with certain input terms regardless of the given input data. This problem seriously degrades the quality of the generation results. In this paper, we propose a novel dynamic topic tracker for solving this problem. Different from existing models, our proposed model learns a global hidden representation for topics and recognizes the corresponding topic during each generation step. The recognized topic is used as additional information to guide the generation process and thus alleviates the offtopic problem. The experimental results show that our proposed model can enhance the performance of sentence generation and the offtopic problem is significantly mitigated.
2019
pdf
abs
Word and Document Embedding with vMFMixture Priors on Context Word Vectors
Shoaib Jameel

Steven Schockaert
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Word embedding models typically learn two types of vectors: target word vectors and context word vectors. These vectors are normally learned such that they are predictive of some word cooccurrence statistic, but they are otherwise unconstrained. However, the words from a given language can be organized in various natural groupings, such as syntactic word classes (e.g. nouns, adjectives, verbs) and semantic themes (e.g. sports, politics, sentiment). Our hypothesis in this paper is that embedding models can be improved by explicitly imposing a cluster structure on the set of context word vectors. To this end, our model relies on the assumption that context word vectors are drawn from a mixture of von MisesFisher (vMF) distributions, where the parameters of this mixture distribution are jointly optimized with the word vectors. We show that this results in word vectors which are qualitatively different from those obtained with existing word embedding models. We furthermore show that our embedding model can also be used to learn highquality document representations.
2018
pdf
abs
Relation Induction in Word Embeddings Revisited
Zied Bouraoui

Shoaib Jameel

Steven Schockaert
Proceedings of the 27th International Conference on Computational Linguistics
Given a set of instances of some relation, the relation induction task is to predict which other word pairs are likely to be related in the same way. While it is natural to use word embeddings for this task, standard approaches based on vector translations turn out to perform poorly. To address this issue, we propose two probabilistic relation induction models. The first model is based on translations, but uses Gaussians to explicitly model the variability of these translations and to encode soft constraints on the source and target words that may be chosen. In the second model, we use Bayesian linear regression to encode the assumption that there is a linear relationship between the vector representations of related words, which is considerably weaker than the assumption underlying translation based models.
pdf
abs
Unsupervised Learning of Distributional Relation Vectors
Shoaib Jameel

Zied Bouraoui

Steven Schockaert
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Word embedding models such as GloVe rely on cooccurrence statistics to learn vector representations of word meaning. While we may similarly expect that cooccurrence statistics can be used to capture rich information about the relationships between different words, existing approaches for modeling such relationships are based on manipulating pretrained word vectors. In this paper, we introduce a novel method which directly learns relation vectors from cooccurrence statistics. To this end, we first introduce a variant of GloVe, in which there is an explicit connection between word vectors and PMI weighted cooccurrence vectors. We then show how relation vectors can be naturally embedded into the resulting vector space.
2017
pdf
abs
Modeling Context Words as Regions: An Ordinal Regression Approach to Word Embedding
Shoaib Jameel

Steven Schockaert
Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)
Vector representations of word meaning have found many applications in the field of natural language processing. Word vectors intuitively represent the average context in which a given word tends to occur, but they cannot explicitly model the diversity of these contexts. Although region representations of word meaning offer a natural alternative to word vectors, only few methods have been proposed that can effectively learn word regions. In this paper, we propose a new word embedding model which is based on SVM regression. We show that the underlying ranking interpretation of word contexts is sufficient to match, and sometimes outperform, the performance of popular methods such as Skipgram. Furthermore, we show that by using a quadratic kernel, we can effectively learn word regions, which outperform existing unsupervised models for the task of hypernym detection.
2016
pdf
abs
DGloVe: A Feasible Least Squares Model for Estimating Word Embedding Densities
Shoaib Jameel

Steven Schockaert
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
We propose a new word embedding model, inspired by GloVe, which is formulated as a feasible least squares optimization problem. In contrast to existing models, we explicitly represent the uncertainty about the exact definition of each word vector. To this end, we estimate the error that results from using noisy cooccurrence counts in the formulation of the model, and we model the imprecision that results from including uninformative context words. Our experimental results demonstrate that this model compares favourably with existing word embedding models.
2012
pdf
Ngram Fragment Sequence Based Unsupervised DomainSpecific Document Readability
Shoaib Jameel

Xiaojun Qian

Wai Lam
Proceedings of COLING 2012