@inproceedings{chen-etal-2026-tasks,
    title = "From Tasks to Teams: A Risk-First Evaluation Framework for Multi-Agent {LLM} Systems in Finance",
    author = "Chen, Zichen  and
      Chen, Jianda  and
      Chen, Jiaao  and
      Sra, Misha",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1934/",
    pages = "38819--38857",
    ISBN = "979-8-89176-395-1",
    abstract = "Current financial benchmarks prioritize large language models (LLMs) for task accuracy and portfolio returns, yet overlook risks arising from multi-agent cooperation, tool-sharing, and real-world financial actions. We introduce M-SAEA, a Multi-agent, Safety-Aware Evaluation Agent that audits LLM teams without fine-tuning, deploying ten probes across four layers: model, workflow, interaction, and system, to yield a continuous risk vector and natural-language rationale. Evaluated across three high-stakes tasks (finance management, webshop automation, transactional services) with six prominent models, M-SAEA (i) identifies unsafe trajectories with minimal false positives, (ii) reveals latent risks (e.g., temporal staleness) that are not addressed by standard metrics, and (iii) provides granular, actionable scores for balancing safety and latency pre-deployment. By quantifying safety as a model-agnostic metric, M-SAEA reorients evaluation from individual tasks to collaborative teams, offering a robust template for risk-first assessment of agentic AI in finance and beyond."
}