
\section{The fruit and tools game}
\label{sec:thegame}

Our game, inspired by Tomasello's \shortcite{Tomasello:2014}
conjecture that the unique cognitive abilities of humans arose from
the requirements of cooperative interaction, is schematically
illustrated in Fig.~\ref{fig::gameexemple}. In each episode, a
randomly selected agent is presented with instances of two tools
(\emph{knife}, \emph{fork}, \emph{axe}\ldots), the other with a fruit
instance (\emph{apple}, \emph{pear}, \emph{plum}\ldots).
Tools and fruits are represented by property vectors (e.g., \emph{has
  a blade}, \emph{is small}), with each instance characterized by
values randomly varying around the category mean (e.g., an apple
instance might be smaller than another). An agent is randomly selected
to be the first to perform an action. The game then proceeds for an
arbitrary number of \emph{turns}. At each turn, one of the agents must
decide whether to pick one of the two tools and stop, or to
continue, in which case the message it utters is passed to the other
agent, and the game proceeds. Currently, for
ease of analysis, messages are single discrete symbols selected from a
vocabulary of size $10$, but extension to symbol sequences is
trivial (although it would of course complicate the analysis). As soon as an agent picks a tool, the game ends. The agents
receive a binary reward of 1 if they picked the better tool for the
fruit at hand, 0 otherwise. The best choice is computed by a utility
function that takes into account the interaction between tool and
fruit instance properties (e.g., as in Fig.~\ref{fig::gameexemple}, a
tool with a round edge might be particularly valuable if the fruit has
a pit). Utility is relative: given a peach, the axe is worse than the
spoon, but it would be the
better tool when the alternative is a hammer.\\
%
\indent Here are some desirable properties of our setup, as a
simplified simulation of human interactions. The agents are fully
symmetric and cannot specialize to a fixed role or turn-taking
scheme. The number of turns is open and determined by the agents. In
pure signaling/referential games \citep{Lewis:1969}, the aim is
successful communication itself. In our game, reward depends instead
on tool and fruit affordances. Optimal performance can only be
achieved by jointly reasoning about the properties of the tools and
how they relate to the fruit. Humans are rewarded when
they use language to solve problems of this sort, and not for
successful acts of reference \emph{per se}. Finally, as we use
commonsense descriptions of everyday objects to build our dataset (see
below), the distribution of their properties possesses the highly
skewed characteristics encountered everywhere in the human environment
\cite{Li:2002}. For example, if the majority of fruits requires to be cut, a knife is intrinsically more useful than a spoon. Note that the agents do not have any a priori knowledge of the tools utility. Yet, baseline agents are able to discover context-independent tool
affordances and already reach high performance. We believe that this
scenario, in which communication-transmitted information complements
knowledge that can be directly inferred by observing the world, is
more interesting than typical games in which
language is the only information carrier.\\
%
\paragraph{Game ingredients and utility} We picked 16 tool and 31 fruit
categories from \newcite{McRae:etal:2005} and
\newcite{Silberer:etal:2013}, who provide subject-elicited
property-based commonsense descriptions of objects, with some
extensions. We used $11$ fruit and $15$ tool features from these
databases to represent the categories. We rescaled the
elicitation-frequency-based property values provided in the norms to
lie in the $[0,1]$ range, and manually changed some counter-intuitive
values. An object instance is a property vector sampled from the
corresponding category as follows. For binary properties such as
\emph{has a pit}, we use Bernoulli sampling with $p$ equaling the
category value. For continuous properties such as \emph{is small}, we
sample uniformly from $[\mu - 0.1,\mu + 0.1]$, where $\mu$ is the
category value. We then devised a function returning an utility score
for any fruit-tool property vector pair. The function maps
properties to a reduced space of abstract functional features (such as
\emph{break} for tools, and \emph{hard} for fruits). Details are in
Supplementary Section \ref{suppsec:data}. For example, an apple with
\emph{is crunchy=0.7} value gets a high \emph{hard}  functional feature
score. A knife with \emph{has a blade=1} gets a high \emph{cut} score,
and therefore high utility for the \emph{hard}
apple. % On the contrary, a spoon which main feature is
% \emph{has a round end=0.8} is not able to \emph{cut} and has a low
% utility for that apple.
Some features, e.g., \emph{has a handle} for tools, have no impact on
utility. They only represent realistic aspects of objects and act as
noise. Our dataset with full category property vectors will be
publicly released along with code.\\
%
\begin{figure*}[t!]
\includegraphics[width=\textwidth]{images/LargeSlide.pdf}
\caption{Two turns of dialogue. Dashed boxes are not used in this game episode due to the agent roles (the blue/left agent is Tool Player, the green/right one is Fruit player). The flow is explained in detail in the text.}
\label{fig:oneround}
\end{figure*}
%
\paragraph{Datasets} We separate the $31$ fruit categories into three
sets: in-domain ($21$ categories), validation and transfer ($5$
categories each). The in-domain set is further split into train and
test partitions. We train agents on the in-domain train partition and
monitor convergence on the validation set. We report test performance
on the in-domain test partition and on the transfer set. For example,
the \emph{peach} category is in-domain, meaning that distinct peach
instances will be seen at training and in-domain testing time. The
\emph{nectarine} category is in the transfer set, so nectarine
instances will only be seen at test time. This scheme tests the
generalization abilities of the agents (that can generalize to new
fruits since they are all are described in the same feature space). We
generate $210,000$ in-domain training samples and $25,000$ samples for
the other sets, balanced across fruits and tools (that are common
across the sets).
