@inproceedings{suizu-etal-2026-automatic,
    title = "Automatic Generation of a Compositional {QA} Benchmark for Geospatial Reasoning under Spatial and Entity Constraints",
    author = "Suizu, Tetsuhisa  and
      Higashiyama, Shohei  and
      Shindo, Hiroyuki  and
      Ouchi, Hiroki  and
      Sakti, Sakriani",
    editor = "Baez Santamaria, Selene  and
      Somayajula, Sai Ashish  and
      Yamaguchi, Atsuki",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 4: Student Research Workshop)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.eacl-srw.61/",
    pages = "818--830",
    ISBN = "979-8-89176-383-8",
    abstract = "Despite their recent success, the geospatial reasoning capabilities of large language models (LLMs){---}which require understanding spatial relationships among real-world geo-entities{---}remain underexplored.We propose an automatic method for constructing compositional geographic question answering datasets that jointly consider spatial and entity constraints.The generated dataset serves as a principled benchmark for evaluating how LLMs coordinate spatial computation with entity-level understanding under diverse compositional settings.We evaluate two state-of-the-art LLMs, GPT-5.2 and Gemini 3 Flash, on our dataset. Experimental results show that while the models perform relatively well on questions involving rich entity grounding, their accuracy drops substantially on questions requiring precise quantitative spatial reasoning, such as distance estimation and containment judgment.Our dataset is publicly available for research and reproduction."
}