@inproceedings{chen-etal-2026-medsyn,
    title = "{MEDSYN}: Benchmarking Multi-{E}vi{D}ence {SYN}thesis in Complex Clinical Cases for Multimodal Large Language Models",
    author = "Chen, Boqi  and
      Liu, Xudong  and
      Peng, Jiachuan  and
      Frey-Marti, Marianne  and
      Lam, Kyle  and
      Zheng, Bang  and
      Li, Lin  and
      Qiu, Jianing",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1183/",
    pages = "23631--23655",
    ISBN = "979-8-89176-395-1",
    abstract = "Multimodal large language models (MLLMs) have shown great potential in medical applications, yet existing benchmarks inadequately capture real-world clinical complexity. We introduce MEDSYN, a multilingual, multimodal benchmark of highly complex clinical cases with up to 7 distinct visual clinical evidence (CE) types per case. Mirroring clinical workflow, we evaluate 18 MLLMs on differential diagnosis (DDx) generation and final diagnosis (FDx) selection. While frontier models often match or even outperform human experts on DDx generation, all MLLMs exhibit a much larger DDx{--}FDx performance gap compared to expert clinicians, indicating a failure mode in synthesis of heterogeneous CE types. Ablations attribute this failure to (i) overreliance on less discriminative textual CE (\textit{e.g.}, medical history) and (ii) a cross-modal CE utilization gap. We introduce \textit{Evidence Sensitivity} to quantify the latter and show that a smaller gap correlates with higher diagnostic accuracy. Finally, we demonstrate how it can be used to guide interventions to improve model performance. https://github.com/jianing-lab/MEDSYN."
}