@inproceedings{sviridov-etal-2025-3mdbench,
    title = "3{MDB}ench: Medical Multimodal Multi-agent Dialogue Benchmark",
    author = "Sviridov, Ivan  and
      Miftakhova, Amina  and
      Vladimirovich, Tereshchenko Artemiy  and
      Zubkova, Galina  and
      Blinov, Pavel  and
      Savchenko, Andrey",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1353/",
    pages = "26625--26665",
    ISBN = "979-8-89176-332-6",
    abstract = "Though Large Vision-Language Models (LVLMs) are being actively explored in medicine, their ability to conduct complex real-world telemedicine consultations combining accurate diagnosis with professional dialogue remains underexplored. This paper presents **3MDBench** (**M**edical **M**ultimodal **M**ulti-agent **D**ialogue **Bench**mark), an open-source framework for simulating and evaluating LVLM-driven telemedical consultations. 3MDBench simulates patient variability through temperament-based Patient Agent and evaluates diagnostic accuracy and dialogue quality via Assessor Agent. It includes 2996 cases across 34 diagnoses from real-world telemedicine interactions, combining textual and image-based data. The experimental study compares diagnostic strategies for widely used open and closed-source LVLMs. We demonstrate that multimodal dialogue with internal reasoning improves F1 score by 6.5{\%} over non-dialogue settings, highlighting the importance of context-aware, information-seeking questioning. Moreover, injecting predictions from a diagnostic convolutional neural network into the LVLM{'}s context boosts F1 by up to 20{\%}. Source code is available at https://github.com/univanxx/3mdbench."
}