@inproceedings{wang-etal-2025-omni,
    title = "Omni-Chart-600{K}: A Comprehensive Dataset of Chart Types for Chart Understanding",
    author = "Wang, Shulei  and
      Yang, Shuai  and
      Lin, Wang  and
      Guo, Zirun  and
      Cai, Sihang  and
      Huang, Hai  and
      Wang, Ye  and
      Chen, Jingyuan  and
      Jin, Tao",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.findings-naacl.226/",
    pages = "4051--4069",
    ISBN = "979-8-89176-195-7",
    abstract = "To address the deficiencies in chart types and the limited scope of chart tasks in existing datasets, we conducted a comprehensive review of current data collection methodologies. By integrating manual annotation with data generation leveraging GPT-4, we developed a dataset that includes 21 diverse chart types and a broad spectrum of tasks, such as data retrieval and mathematical reasoning. Our analysis of existing models revealed that capabilities in information extraction, mathematical reasoning, and understanding of multiple chart types are essential for performing a variety of chart tasks. To overcome the limitations in these areas, we devised a two-stage training strategy and a method for jointly training the vision encoder tailored for multi-type charts. In the first stage, we designed several tasks to enhance the model{'}s general understanding of charts, aligning multimodal large models pre-trained on natural images to chart tasks. To further improve the model{'}s capability to understand various chart tasks and enhance its reasoning abilities, we employed Chain-of-Thought data for training in the second stage. Through two-stage training on our proposed dataset, the pre-trained multimodal large language model achieved state-of-the-art performance across multiple chart understanding tasks, demonstrating the superiority of our data and methods."
}