Oriol Vinyals


2021

pdf
Machine Translation Decoding beyond Beam Search
Rémi Leblond | Jean-Baptiste Alayrac | Laurent Sifre | Miruna Pislar | Lespiau Jean-Baptiste | Ioannis Antonoglou | Karen Simonyan | Oriol Vinyals
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Beam search is the go-to method for decoding auto-regressive machine translation models. While it yields consistent improvements in terms of BLEU, it is only concerned with finding outputs with high model likelihood, and is thus agnostic to whatever end metric or score practitioners care about. Our aim is to establish whether beam search can be replaced by a more powerful metric-driven search technique. To this end, we explore numerous decoding algorithms, including some which rely on a value function parameterised by a neural network, and report results on a variety of metrics. Notably, we introduce a Monte-Carlo Tree Search (MCTS) based method and showcase its competitiveness. We provide a blueprint for how to use MCTS fruitfully in language applications, which opens promising future directions. We find that which algorithm is best heavily depends on the characteristics of the goal metric; we believe that our extensive experiments and analysis will inform further research in this area.

pdf
WikiGraphs: A Wikipedia Text - Knowledge Graph Paired Dataset
Luyu Wang | Yujia Li | Ozlem Aslan | Oriol Vinyals
Proceedings of the Fifteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-15)

We present a new dataset of Wikipedia articles each paired with a knowledge graph, to facilitate the research in conditional text generation, graph generation and graph representation learning. Existing graph-text paired datasets typically contain small graphs and short text (1 or few sentences), thus limiting the capabilities of the models that can be learned on the data. Our new dataset WikiGraphs is collected by pairing each Wikipedia article from the established WikiText-103 benchmark (Merity et al., 2016) with a subgraph from the Freebase knowledge graph (Bollacker et al., 2008). This makes it easy to benchmark against other state-of-the-art text generative models that are capable of generating long paragraphs of coherent text. Both the graphs and the text data are of significantly larger scale compared to prior graph-text paired datasets. We present baseline graph neural network and transformer model results on our dataset for 3 tasks: graph -> text generation, graph -> text retrieval and text -> graph retrieval. We show that better conditioning on the graph provides gains in generation and retrieval quality but there is still large room for improvement.

2016

pdf bib
Generating Sentences from a Continuous Space
Samuel R. Bowman | Luke Vilnis | Oriol Vinyals | Andrew Dai | Rafal Jozefowicz | Samy Bengio
Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning

pdf
Multilingual Language Processing From Bytes
Dan Gillick | Cliff Brunk | Oriol Vinyals | Amarnag Subramanya
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2015

pdf
Sentence Compression by Deletion with LSTMs
Katja Filippova | Enrique Alfonseca | Carlos A. Colmenares | Lukasz Kaiser | Oriol Vinyals
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

pdf bib
Addressing the Rare Word Problem in Neural Machine Translation
Thang Luong | Ilya Sutskever | Quoc Le | Oriol Vinyals | Wojciech Zaremba
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)