@inproceedings{wang-etal-2025-ecom,
    title = "{EC}om-Bench: Can {LLM} Agent Resolve Real-World {E}-commerce Customer Support Issues?",
    author = "Wang, Haoxin  and
      Peng, Xianhan  and
      Cheng, Huang  and
      Huang, Yizhe  and
      Gong, Ming  and
      Yang, Chenghan  and
      Liu, Yang  and
      Lin, Jiang",
    editor = "Potdar, Saloni  and
      Rojas-Barahona, Lina  and
      Montella, Sebastien",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track",
    month = nov,
    year = "2025",
    address = "Suzhou (China)",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-industry.19/",
    pages = "276--284",
    ISBN = "979-8-89176-333-3",
    abstract = "In this paper, we introduce , the first benchmark framework for evaluating LLM agent with multimodal capabilities in the e-commerce customer support domain. ECom-Bench features dynamic user simulation based on persona information collected from real e-commerce customer interactions and a realistic task dataset derived from authentic e-commerce dialogues. These tasks, covering a wide range of business scenarios, are designed to reflect real-world complexities, making highly challenging. For instance, even advanced models like GPT-4o achieve only a 10{--}20{\%} pass3 metric in our benchmark, highlighting the substantial difficulties posed by complex e-commerce scenarios. The code and data have been made publicly available at \url{https://github.com/XiaoduoAILab/ECom-Bench} to facilitate further research and development in this domain."
}