\subsection{Performance and pragmatics}
\label{sec:prag}

We report mean and standard error of the mean (SEM) over successful training seeds.\footnote{That is, training seeds leading to final validation performance above $85\%$.} % At test-time we take the most probable messages and choices from the agents.
Each agent A or B can be either $F$ (Fruit Player) or $T$ (Tool Player) and in position $1$ or $2$, depending on the test game. We measure to what extent Tool Player influences  Fruit Player ($\text{ME}^{T\rightarrow F}$) and \emph{vice versa} ($\text{ME}^{F\rightarrow T}$). Similarly, we evaluate position impact by computing $\text{ME}^{1\rightarrow 2}$ and $\text{ME}^{2\rightarrow 1}$. We average ME values over messages sent during each test game, and report averages over test games. Note that we also intervene on the dummy initialization message used at $t=0$, which is received by the agent in position $1$. This impacts the value of $\text{ME}^{2\rightarrow 1}$. If the agent in position $1$ has learned to rely on the initialization message to understand that the game is beginning, an intervention on this message will have an influence we want to take into account.\footnote{Conversely, we  ignore the messages agents send when stopping the game, as they are never heard.} Similarly, in the no-communication ablation, when computing ME values, we replace the dummy fixed message the agents receive with a counterfactual. Finally, we emphasize that the computation of ME values does not interfere with game dynamics and does not affect performance.

\begin{table*}[t!]
  \begin{center}
\begin{tabular}{|c|c|cc|cc|}
  \hline
  ~& ~&\multicolumn{2}{c|}{\bfseries No communication} &\multicolumn{2}{c|}{\bfseries With communication}\\
\hline
~ & Metric & In & Transfer & In & Transfer \\
\hline
\multirow{ 7}{*}{\rotatebox[origin=c]{90}{\bfseries No memory}}&
Av. perf. (\%)& $84.83\pm0.09$& $84.0\pm0.11$&$96.9\pm0.32$& $94.5\pm0.37$\\
~&$ME^{F\rightarrow T}$& $0.133^*\pm0.01$& $0.14^*\pm0.01$&$5.0^*\pm0.39$& $5.0^*\pm0.36$\\
~&$ME^{T\rightarrow F}$& $0.05\pm0.02$& $0.030\pm0.01$&$3.9\pm0.38$& $3.3\pm0.30$\\
~&$ME^{1\rightarrow 2}$& $0.066\pm0.00$& $0.067\pm0.01$&$3.9\pm0.29$& $3.7\pm0.26$\\
~&$ME^{2\rightarrow 1}$& $0.12^*\pm0.02$& $0.10^*\pm0.01$&$5.0^*\pm0.38$& $4.7^*\pm0.33$\\
~&Bi. comm. (\%)& $1.4\pm0.31$& $1.3\pm0.40$&$\bm{88\pm2.49}$& $\bm{89\pm2.24}$\\
~&Av. conv. length& $0.508\pm0.01$& $0.52\pm0.01$&$2.16\pm0.08$& $2.21\pm0.10$\\
~&T chooses (\%)&$99.4\pm0.63$& $99.6\pm0.56$&$85\pm2.09$& $83\pm2.56$\\
\hline
\multirow{ 7}{*}{\rotatebox[origin=c]{90}{\bfseries With memory}}&
Av. perf. (\%)& $88.5\pm0.11$& $87.7\pm0.16$&$\bm{97.4\pm0.12}$& $\bm{95.3\pm0.16}$\\
~&$ME^{F\rightarrow T}$& $0.11^*\pm0.01$& $0.13^*\pm0.01$&$3.0^*\pm0.29$& $2.8^*\pm0.24$\\
~&$ME^{T\rightarrow F}$& $0.064\pm0.01$& $0.071\pm0.01$&$1.8\pm0.22$& $1.8\pm0.21$\\
~&$ME^{1\rightarrow 2}$& $0.085\pm0.01$& $0.10\pm0.01$&$2.4\pm0.29$& $2.3\pm0.22$\\
~&$ME^{2\rightarrow 1}$& $0.093\pm0.01$& $0.103\pm0.01$&$2.4\pm0.22$& $2.4\pm0.21$\\
~&Bi. comm. (\%)& $3.8\pm0.61$& $4.6\pm0.68$& $78\pm2.55$& $78\pm2.65$\\
~&Av. conv. length& $1.50\pm0.06$& $1.46\pm0.06$&$2.7\pm0.11$& $2.7\pm0.11$\\
~&T chooses (\%)&$87.3\pm1.34$& $85.8\pm1.48$&$81\pm2.94$& $81\pm3.00$\\
\hline
\end{tabular}
\caption{Test performance and pragmatic measures mean and SEM in different settings. ``Av.~perf.'' (average performance) denotes $\%$ of samples where best tool was chosen, ``Bi.~comm." denotes $\%$ of games with bilateral communication taking place. ``Av.~conv.~length" is average conversation length in turns. ``T~chooses" denotes $\%$ of games ended by Tool Player. Values of $\text{ME}$ with an asterisk $~^*$ are statistically significantly higher than their reverse (e.g. $\text{ME}^{F\rightarrow T}>\text{ME}^{T\rightarrow F}$). Best ``Av.~perf.'' and ``Bi.~comm." in bold.}
\label{tab:perftable}
\end{center}
\end{table*}

\paragraph{Both communication and memory help} Table \ref{tab:perftable} shows that enabling the agents to communicate greatly increases performance compared to the no-communication ablation, both with and without memory, despite the high baseline set by agents that learn about tool usefulness without communicating (see discussion below). Agents equipped with memory  perform better than their no-memory counterparts, but the gain in performance is smaller compared to the gain attained from communication.  The overall best performance is achieved with communication \emph{and} memory. We also see that the agents generalize well, with transfer-fruit performance almost matching that on in-domain fruits. Next, we analyze in detail the impact of each factor (communication, memory) on agent performance and strategies.\\
%
\paragraph{No-communication, no-memory} We start by looking at how the
game unfolds when communication and agent memory are ablated (top left
quadrant of Table \ref{tab:perftable}). Performance is largely above
chance ($\approx{}50\%$), because, as discussed in Section
\ref{sec:thegame}, some tools are intrinsically better on average
across fruits than others. % We could have removed this bias, but we
% preferred not, since a similar skewness seems to be a basic fact of
% the human environment in which language emerged.
Without
communication, the agents exploit this bias and learn a strategy where
(i) Fruit Player never picks the tool but always continues the
game and (ii) Tool Player picks the tool according to average tool
usefulness. Indeed, Tool
Player makes the choice in more than $99\%$ of the games. Conversation length is $0$ if Tool Player starts and $1$ if it is
the second agent, requiring the starting Fruit Player to pass its
turn.  Reassuringly, \text{ME} values are low, confirming the
reliability of this communication score, and indicating that
communication-deprived agents did not learn to rely on the fixed dummy
message (e.g., by using it as a constant bias). Still, we observe
that, across the consistently low values, Fruit Player appears to
affect Tool Player
significantly more than the reverse ($\text{ME}^{F\rightarrow T}>\text{ME}^{T\rightarrow F}$). This is generally observed in all configurations, and we believe it due to the fact that Tool Player takes charge of most of the reasoning in the game. We come back to this later in our analysis. We also observe that the second player impacts the first more than the reverse ($\text{ME}^{2\rightarrow 1} > \text{ME}^{1\rightarrow 2}$). We found this to be an artifact of the strategy adopted by the agents. In the games in which Tool Player starts and immediately stops the game, we can only compute ME for the Tool/position-1 agent, by intervening on the initialization. The resulting value, while tiny, is unlikely to be exactly $0$. In the games where Fruit Player starts and Tool Player stops at the second turn, we compute instead two tiny MEs, one per agent. Hence, the observed asymmetry. We verified this hypothesis by removing single-turn games: the influence of the second player on the first indeed disappears.\\
%
\paragraph{Impact of communication} The top quadrants of
Table \ref{tab:perftable} show that communication helps performance, despite the high baseline set by the
``average tool usefulness'' strategy.  Importantly, when communication
is added, we see a dramatic increase in the proportion of games
with bilateral communication, confirming that
improved performance is not due to an accidental effect of adding a new channel \cite{Lowe:etal:2019}. ME and average number of turns
also increase. Fruit Player is the more influential agent. This effect is not due to the artifact we found in the no-communication ablation, because almost all conversations, including those started by Tool Player, are longer than one turn, so we can compute  both $\text{ME}^{F\rightarrow T}$ and $\text{ME}^{T\rightarrow F}$. We believe the asymmetry to be due to the fact that  Tool Player is the agent that demands more information from the other, as it is the one that sees the tools, and that in the large majority of cases makes the final choice. Supplementary Table \ref{tab:perftable2} shows that the gap between the influence of the Fruit Player on the Tool player and its reverse is greater when the Fruit Player is in position $2$. This, then, explains $\text{ME}^{2\rightarrow 1} > \text{ME}^{1\rightarrow 2}$ as an epiphenomenon of Fruit Player being more influential.\\
% Supplementary Table \ref{tab:perftable2} shows that the Fruit Player has greater influence only when in position $2$.
%
\paragraph{Is memory ablation necessary for communication to matter?}
An important observation from previous research is that depriving at
least one agent of memory might be necessary to develop successful
multi-turn communication
\cite{Kottur:etal:2017,Cao:etal:2018,Evtimova:etal:2018}. This is
undesirable, as obviously language should not emerge simply as a
surrogate for memories of amnesiac agents. The performance and
communicative behaviours results in the bottom right quadrant of Table
\ref{tab:perftable} show that, in our game, genuine linguistic
interaction (as cued by ME and bilateral communication scores) is
present even when both agents are equipped with memory. It is
interesting however to study how adding memory affects the game
dynamics independently of communication. In the bottom left quadrant,
we see that memory leads to some task performance improvement for
communication-less agents. Manual inspection of example games reveals
that such agents are developing turn-based strategies.  For example,
Tool Player learns to continue the game at turn $t$ if $tool_1$ has a
round end. At $t+1$, Fruit Player can use the fact that Tool Player
continues at $t$ as information about relative tool roundness, and
either pick the appropriate one based on the fruit or continue to
gather more information. In a sense, agents learn to use the
possibility to stop or continue at each turn as a rudimentary
communication channel. Indeed, exchanges are on average longer when
memory is involved, and turn-based strategies appear even with
communication. In the latter case, agents rely on communication but also on turn-based
schemes, resulting in lower \text{ME} values and bilateral
communication compared to the no-memory ablation. Finally, the
respective positions of the agents in the conversation no longer
impact ME
($\text{ME}^{1\rightarrow 2}\approx{}\text{ME}^{2\rightarrow
  1}$). This might be because, with memory, the starting agent can
identify whether it is at turn $t=0$, where it almost always chooses
to continue the game to send and receive more information via
communication. Intervening on the dummy initialization message has a
lower influence, resulting in lower $\text{ME}^{2\rightarrow 1}$.
