@inproceedings{barbi-etal-2025-preventing,
    title = "Preventing Rogue Agents Improves Multi-Agent Collaboration",
    author = "Barbi, Ohav  and
      Yoran, Ori  and
      Geva, Mor",
    editor = "Kamalloo, Ehsan  and
      Gontier, Nicolas  and
      Lu, Xing Han  and
      Dziri, Nouha  and
      Murty, Shikhar  and
      Lacoste, Alexandre",
    booktitle = "Proceedings of the 1st Workshop for Research on Agent Language Models (REALM 2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-felix-schneider-kit/2025.realm-1.34/",
    pages = "486--511",
    ISBN = "979-8-89176-264-0",
    abstract = "Multi-agent systems, where specialized agents collaborate to solve a shared task hold great potential, from increased modularity to simulating complex environments. However, they also have a major caveat {--} a single agent can cause the entire system to fail. Consider a simple game where the knowledge to solve the task is distributed between agents, which share information in a communication channel. At each round, any of the agents can terminate the game and make the final prediction, even if they are uncertain about the outcome of their action. Detection of such rogue agents {\_}{\_}before they act{\_}{\_} may prevent the system{'}s failure. In this work, we propose to {\_}{\_}monitor{\_}{\_} agents during action prediction and {\_}{\_}intervene{\_}{\_} when a future error is likely to occur. To test our approach, we introduce WhoDunitEnv, a multi-agent collaboration environment that allows modular control over task complexity and communication structure. Experiments on WhoDunitEnv, code generation tasks and the GovSim environment for resource sustainability show that our approach leads to substantial performance gains up to 17.4{\%}, 2.5{\%} and 20{\%}, respectively. Thorough analysis shows that our monitors successfully identify critical points of agent confusion and our interventions effectively stop agent errors from propagating."
}