@inproceedings{khalighinejad-etal-2025-matvix,
    title = "{M}at{V}i{X}: Multimodal Information Extraction from Visually Rich Articles",
    author = "Khalighinejad, Ghazal  and
      Scott, Sharon  and
      Liu, Ollie  and
      Anderson, Kelly L.  and
      Stureborg, Rickard  and
      Tyagi, Aman  and
      Dhingra, Bhuwan",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/moar-dois/2025.naacl-long.185/",
    doi = "10.18653/v1/2025.naacl-long.185",
    pages = "3636--3655",
    ISBN = "979-8-89176-189-6",
    abstract = "Multimodal information extraction (MIE) is crucial for scientific literature, where valuable data is often spread across text, figures, and tables. In materials science, extracting structured information from research articles can accelerate the discovery of new materials. However, the multimodal nature and complex interconnections of scientific content present challenges for traditional text-based methods. We introduce MatViX, a benchmark consisting of 324 full-length research articles and 1,688 complex structured JSON files, carefully curated by domain experts in polymer nanocomposites and biodegradation. These JSON files are extracted from text, tables, and figures in full-length documents, providing a comprehensive challenge for MIE. We introduce a novel evaluation method to assess the accuracy of curve similarity and the alignment of hierarchical structures. Additionally, we benchmark vision-language models (VLMs) in a zero-shot manner, capable of processing long contexts and multimodal inputs. Our results demonstrate significant room for improvement in current models."
}