@inproceedings{chen-etal-2026-browsecomp,
    title = "{B}rowse{C}omp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents",
    author = "Chen, Zijian  and
      Ma, Xueguang  and
      Zhuang, Shengyao  and
      Nie, Ping  and
      Zou, Kai  and
      Sharifymoghaddam, Sahel  and
      Liu, Andrew  and
      Green, Joshua  and
      Patel, Kshama  and
      Meng, Ruoxi  and
      Su, Mingyi  and
      Li, Yanxi  and
      Hong, Haoran  and
      Shi, Xinyu  and
      Liu, Xuye  and
      Oyarhoseini, Hosna  and
      Thakur, Nandan  and
      Zhang, Crystina  and
      Gao, Luyu  and
      Chen, Wenhu  and
      Lin, Jimmy",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1023/",
    pages = "22349--22370",
    ISBN = "979-8-89176-390-6",
    abstract = "Deep search agents that combine large language models with retrieval tools excel at complex, multi-hop queries. Yet, existing benchmarks such as BrowseComp rely on black-box web search APIs, facing key limitations. (1) Fairness: for agents, dynamic and opaque web APIs hinder reproducibility and fair comparisons across agents. (2) Disentanglement: for retrieval, the lack of a fixed document corpus makes it impossible to isolate retriever contributions from end-to-end search agent accuracy. We introduce BrowseComp-Plus, a benchmark derived from BrowseComp that employs a fixed, human-verified corpus, enabling controlled retrieval for deep search agents. BrowseComp-Plus clearly distinguishes agent performance: with a BM25 retriever, the open-source Search-R1 achieves 3.86{\%} accuracy, while GPT-5 achieves 55.9{\%}. Additionally, BrowseComp-Plus makes retrieval gains explicit: pairing GPT-5 with Qwen3-Embedding-8B retriever further improves accuracy to 70.1{\%} while reducing search calls. Overall, BrowseComp-Plus provides a fair and disentangled testbed, advancing both deep search agent evaluation and retrieval research for agentic search. Code and data can be found at: https://texttron.github.io/BrowseComp-Plus/"
}