@inproceedings{kartha-etal-2026-dashboardqa,
    title = "{D}ashboard{QA}: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards",
    author = "Kartha, Aaryaman  and
      Masry, Ahmed  and
      Islam, Mohammed Saidul  and
      Lang, Thinh  and
      Rahman, Shadikur  and
      Mahbub, Ridwan  and
      Rahman, Mizanur  and
      Ahmed, Mahir  and
      Parvez, Md Rizwan  and
      Hoque, Enamul  and
      Joty, Shafiq",
    editor = "Demberg, Vera  and
      Inui, Kentaro  and
      Marquez, Llu{\'i}s",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EACL} 2026",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.177/",
    pages = "3385--3407",
    ISBN = "979-8-89176-386-9",
    abstract = "Dashboards are powerful visualization tools for data-driven decision-making, integrating multiple interactive views that allow users to explore, filter, and navigate data. Unlike static charts, dashboards support rich interactivity, which is essential for uncovering insights in real-world analytical workflows. However, existing question-answering benchmarks for data visualizations largely overlook this interactivity, focusing instead on static charts. This limitation severely constrains their ability to evaluate the capabilities of modern multimodal agents designed for GUI-based reasoning. To address this gap, we introduce DashboardQA, the first benchmark explicitly designed to assess how vision-language GUI agents comprehend and interact with real-world dashboards. The benchmark includes 292 tasks on 112 interactive dashboards, encompassing 405 question answer pairs overall. These questions span five categories: multiple-choice, factoid, hypothetical, multi-dashboard, and conversational. By assessing a variety of leading closed- and open-source GUI agents, our analysis reveals their key limitations, particularly in grounding dashboard elements, planning interaction trajectories, and performing reasoning. Our findings indicate that interactive dashboard reasoning is a challenging task overall for all the VLMs evaluated. Even the top-performing agents struggle; for instance, the best agent based on Gemini-Pro-2.5 achieves only 38.69{\%} accuracy, while the OpenAI CUA agent reaches just 22.69{\%}, demonstrating the benchmark{'}s significant difficulty. We release DashboardQA at .."
}