Mila > Team > Marc G. Bellemare

# Marc G. Bellemare

Core Industry Member
bellemam@mila.quebec
http://www.marcgbellemare.info/
Deep Learning
Generative Models
Information Theory
Online Learning
Reinforcement Learning (RL)
Core Industry Member
Adjunct Professor, Assistant Professor, McGill University, Université de Montréal, Reliant AI, Canada CIFAR AI Chair

I am the Chief Scientific Officer at Reliant AI. Previously, I was a research scientist at Google Brain in Montréal, Canada focusing on the reinforcement learning effort there. From 2013 to 2017, I was at DeepMind in the UK. I received my Ph.D. from the University of Alberta working with Michael Bowling and Joel Veness.

My research lies at the intersection of reinforcement learning and probabilistic prediction. I’m also interested in deep learning, generative modelling, online learning, and information theory.

## Publications

### 2021-08

Deep Reinforcement Learning at the Edge of the Statistical Precipice
Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C. Courville and Marc G. Bellemare
NEURIPS 2021
(2021-08-30)
proceedings.neurips.cc[LATEST on arXiv: Learning (2021-08-30)]

### 2021-05

Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning
Rishabh Agarwal, Marlos C. Machado, Pablo Samuel Castro and Marc G Bellemare
ICLR 2021
(2021-05-03)
research.google PDF[Also on arXiv preprint arXiv:2101.05265 (2021-01-13)]
The Importance of Pessimism in Fixed-Dataset Policy Optimization
Jacob Buckman, Carles Gelada and Marc G Bellemare
ICLR 2021
(2021-05-03)
dblp.uni-trier.de PDF[Also on arXiv preprint arXiv:2009.06799 (2020-09-15)]

### 2021-02

Metrics and continuity in reinforcement learning
Charline Le Lan, Marc G. Bellemare and Pablo Samuel Castro
AAAI 2021
(2021-02-02)
ojs.aaai.org PDF[LATEST on arXiv preprint arXiv:2102.01514 (2021-02-02)]

### 2020-12

Autonomous navigation of stratospheric balloons using reinforcement learning.
Marc G. Bellemare, Salvatore Candido, Pablo Samuel Castro, Jun Gong, Marlos C. Machado, Subhodeep Moitra, Sameera S. Ponda and Ziyu Wang
Nature
(2020-12-02)
www-nature-com-443.webvpn.bjmu.tsg211.com

### 2020-07

Representations for Stable Off-Policy Reinforcement Learning
Dibya Ghosh and Marc Bellemare
ICML 2020
(2020-07-12)
proceedings.mlr.press PDF[Also on arXiv preprint arXiv:2007.05520 (2020-07-10)]

### 2020-06

The Value-Improvement Path: Towards Better Representations for Reinforcement Learning
Will Dabney, André Barreto, Mark Rowland, Robert Dadashi, John Quan, Marc G. Bellemare and David Silver
AAAI 2020
(2020-06-03)
ojs.aaai.org[LATEST on arXiv preprint arXiv:2006.02243 (2020-06-03)]
A Distributional Analysis of Sampling-Based Reinforcement Learning Algorithms.
Philip Amortila, Doina Precup, Prakash Panangaden and Marc G. Bellemare
AISTATS 2020
(2020-06-03)
proceedings.mlr.press PDF[Also on arXiv preprint arXiv:2003.12239 (2020-03-27)]

### 2020-04

On Bonus Based Exploration Methods In The Arcade Learning Environment
Adrien Ali Taiga, William Fedus, Marlos C. Machado, Aaron Courville and Marc G. Bellemare
ICLR 2020
(2020-04-30)
iclr.cc PDF[LATEST on arXiv preprint arXiv:2109.11052 (2021-09-22)]
Count-Based Exploration with the Successor Representation
Marlos C. Machado, Marc G. Bellemare and Michael Bowling
AAAI 2020
(2020-04-03)
aaai.org PDF
Algorithmic Improvements for Deep Reinforcement Learning Applied to Interactive Fiction.
Vishal Jain, William Fedus, Hugo Larochelle, Doina Precup and Marc G. Bellemare
AAAI 2020
(2020-04-03)
aaai.org PDF[Also on arXiv preprint arXiv:1911.12511 (2019-11-28)]

### 2020-03

Zooming for Efficient Model-Free Reinforcement Learning in Metric Spaces
Ahmed Touati, Adrien Ali Taiga and Marc G. Bellemare
arXiv preprint arXiv:2003.04069
(2020-03-09)
ui.adsabs.harvard.edu PDF
The Hanabi Challenge: A New Frontier for AI Research
Nolan Bard, Jakob N. Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H. Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, Iain Dunning, Shibl Mourad, Hugo Larochelle, Marc G. Bellemare and Michael Bowling
Artificial Intelligence
(2020-03-01)
www.sciencedirect.com PDF

### 2020-01

Shaping the Narrative Arc: Information-Theoretic Collaborative DialoguePaper type: Technical Paper.
Kory Wallace Mathewson, Pablo Samuel Castro, Colin Cherry, George F. Foster and Marc G. Bellemare
ICCC
(2020-01-01)
dblp.uni-trier.de PDF
Shaping the Narrative Arc: Information-Theoretic Collaborative Dialogue
Kory Mathewson, Pablo Samuel Castro, Colin Cherry, George Foster and Marc G. Bellemare
(venue unknown)
(2020-01-01)
On Catastrophic Interference in Atari 2600 Games
William Fedus, Dibya Ghosh, John D. Martin, Marc G. Bellemare, Yoshua Bengio and Hugo Larochelle
arXiv preprint arXiv:2002.12499
(2020-01-01)
ui.adsabs.harvard.edu PDF

### 2019-08

Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment
Adrien Ali Taïga, William Fedus, Marlos C. Machado, Aaron C. Courville and Marc G. Bellemare
arXiv preprint arXiv:1908.02388
(2019-08-06)
ui.adsabs.harvard.edu PDF
An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents.
Felipe Petroski Such, Vashisht Madhavan, Rosanne Liu, Rui Wang, Pablo Samuel Castro, Yulun Li, Jiale Zhi, Ludwig Schubert, Marc G. Bellemare, Jeff Clune and Joel Lehman
IJCAI 2019
(2019-08-01)
www.ijcai.org PDF[Also on arXiv preprint arXiv:1812.07069 (2018-12-17)]

### 2019-07

Off-Policy Deep Reinforcement Learning by Bootstrapping the Covariate Shift
Carles Gelada and Marc G. Bellemare
AAAI 2019
(2019-07-17)
ojs.aaai.org PDF[Also on arXiv preprint arXiv:1901.09455 (2019-01-27)]
A Comparative Analysis of Expected and Distributional Reinforcement Learning
Clare Lyle, Marc G. Bellemare and Pablo Samuel Castro
AAAI 2019
(2019-07-17)
aaai.org PDF

### 2019-06

DeepMDP: Learning Continuous Latent Space Models with Theoretical Guarantees
Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum and Marc Bellemare
ICML 2019
(2019-06-09)
icml.cc

### 2019-05

Statistics and Samples in Distributional Reinforcement Learning.
Mark Rowland, Robert Dadashi, Saurabh Kumar, Remi Munos, Marc G. Bellemare and Will Dabney
ICML 2019
(2019-05-24)
proceedings.mlr.press PDF[Also on arXiv preprint arXiv:1902.08102 (2019-02-21)]
DeepMDP: Learning Continuous Latent Space Models for Representation Learning
Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum and Marc G. Bellemare
ICML 2019
(2019-05-24)
proceedings.mlr.press PDF[LATEST on arXiv preprint arXiv:1906.02736 (2019-06-06)]
The Value Function Polytope in Reinforcement Learning
Robert Dadashi, Adrien Ali Taiga, Nicolas Le Roux, Dale Schuurmans and Marc G. Bellemare
ICML 2019
(2019-05-24)
proceedings.mlr.press PDF[Also on arXiv preprint arXiv:1901.11524 (2019-01-31)]
Task-Agnostic Reinforcement Learning (TARL)
Danijar Hafner, Deepak Pathak, Frederik Ebert, Marc G Bellemare, Raia Hadsell, Rowan McAllister, Amy Zhang, Joelle Pineau, Ahmed Touati and Roberto Calandra
ICLR 2019
(2019-05-06)
iclr.cc

### 2019-04

Distributional reinforcement learning with linear function approximation
Marc G. Bellemare, Nicolas Le Roux, Pablo Samuel Castro and Subhodeep Moitra
AISTATS 2019
(2019-04-11)
proceedings.mlr.press PDF[Also on arXiv preprint arXiv:1902.03149 (2019-02-08)]

### 2019-03

An Introduction to Deep Reinforcement Learning
Vincent François-Lavet, Peter Henderson, Riashat Islam, Marc G. Bellemare and Joelle Pineau
(venue unknown)
(2019-03-31)
nowpublishers.com PDF[Also on arXiv preprint arXiv:1811.12560 (2018-11-30)]

### 2019-02

Hyperbolic Discounting and Learning over Multiple Horizons
William Fedus, Carles Gelada, Yoshua Bengio, Marc G. Bellemare and Hugo Larochelle
arXiv preprint arXiv:1902.06865
(2019-02-19)
ui.adsabs.harvard.edu PDF

### 2019-01

Shaping the Narrative Arc: An Information-Theoretic Approach to Collaborative Dialogue.
Kory Wallace Mathewson, Pablo Samuel Castro, Colin Cherry, George F. Foster and Marc G. Bellemare
arXiv preprint arXiv:1901.11528
(2019-01-31)
ui.adsabs.harvard.edu PDF
A Geometric Perspective on Optimal Representations for Reinforcement Learning
Marc G. Bellemare, Will Dabney, Robert Dadashi, Adrien Ali Taïga, Pablo Samuel Castro, Nicolas Le Roux, Dale Schuurmans, Tor Lattimore and Clare Lyle
NEURIPS 2019
(2019-01-31)
papers.nips.cc PDF[LATEST on arXiv preprint arXiv:1901.11530 (2019-01-31)]
A Comparative Analysis of Expected and Distributional Reinforcement Learning
Clare Lyle, Pablo Samuel Castro and Marc G. Bellemare
arXiv preprint arXiv:1901.11084
(2019-01-30)
ui.adsabs.harvard.edu PDF
Temporally Extended Metrics for Markov Decision Processes.
Philip Amortila, Marc G. Bellemare, Prakash Panangaden and Doina Precup
AAAI 2019
(2019-01-01)
dblp.uni-trier.de PDF

### 2018-12

An Introduction to Deep Reinforcement Learning
Vincent François-Lavet, Peter Henderson, Riashat Islam, Marc G. Bellemare and Joelle Pineau
(venue unknown)
(2018-12-20)
nowpublishers.com PDF

### 2018-11

The Barbados 2018 List of Open Issues in Continual Learning.
Tom Schaul, Hado van Hasselt, Joseph Modayil, Martha White, Adam White, Pierre-Luc Bacon, Jean Harb, Shibl Mourad, Marc G. Bellemare and Doina Precup
arXiv preprint arXiv:1811.07004
(2018-11-16)
ui.adsabs.harvard.edu PDF

### 2018-08

Approximate Exploration through State Abstraction.
Adrien Ali Taïga, Aaron C. Courville and Marc G. Bellemare
arXiv preprint arXiv:1808.09819
(2018-08-29)
dblp.uni-trier.de PDF

Publications collected and formatted using Paperoni