# <img width="8%" src="bot.png"/> Beat The Bot

### A Game for Agents that learn interactively


Dialogue research today functions almost entirely based on extensive supervised learning from humans talking to one another
— usually crowdsourced or publicly available on the internet. This data can differ significantly in distribution from the
environment in which a chatbot might be deployed. To help researchers further explore and push dialogue research forward,
it’s important to have agents out in the real world actually conversing with humans.

To that end, we’ve released a new data collection and model evaluation tool, a Messenger-based Chatbot game
called Beat the Bot, which allows people to interact directly with bots and other humans in real time,
creating rich examples to help train models. Our goal with sharing this new tool is to provide researchers
with high-signal data from live interactions instead of fixed language data. We plan to continuously
enhance this tool’s capabilities (for instance, adding image understanding) to help both improve our
latest dialogue models and further explore dialogue research.

<p align="center"><img width="90%" src="beatthebot.gif" /></p>



If you send a message to the page, you will be matched with a bot and another person.
Both you and the other person will see two responses for every message you send — one from your human partner and one
from a bot. You’ll choose which response is better and continue the conversation from there. The goal is to get your
human speaking partner to choose your message more often than the bot’s. This allows for supervision in two senses:
It provides both the human-human dialogue turns, and a human’s assessment on when the bot fails to match human performance.
We ask users to play a character in a game that is completely disconnected from their personal information.
With user permission at the beginning of the game, the data collected will be open-sourced to facilitate new research
directions for the entire community.


## Messenger Link

Currently closed! Check back at a later date.
