@inproceedings{kao-etal-2026-towards,
    title = "Towards {LLM} Agents for Earth Observation",
    author = "Kao, Chia Hsiang  and
      Zhao, Wenting  and
      Lam, Cheryl  and
      Umap, Aarush  and
      Revankar, Shreelekha  and
      Speas, Samuel  and
      Bhagat, Snehal  and
      Datta, Rajeev  and
      Phoo, Cheng Perng  and
      Mall, Utkarsh  and
      Vondrick, Carl  and
      Bala, Kavita  and
      Hariharan, Bharath",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.124/",
    pages = "2597--2611",
    ISBN = "979-8-89176-395-1",
    abstract = "Earth Observation (EO) provides critical planetary data for environmental monitoring, disaster management, climate science, and other scientific domains. In this work we ask: Are AI systems ready for reliable Earth Observation? To answer this, we introduce **UnivEARTH**, a coding benchmark of 408 yes/no questions from NASA Earth Observatory articles across 7 various topics and over 15 satellite instruments and sources. Using Google Earth Engine API as a tool in a zero-shot setup, LLM agents achieve an accuracy of 40.0{\%} where the code fails to run over 44{\%} of the time. To better understand LLM agent behavior, we also analyze the impact of using the JavaScript API versus Python and the effect of providing documentation. Furthermore, we find that using a reflexion framework significantly reduces errors: Claude-4.5-Sonnet, Gemini-2.5-Pro, and GPT-5 accuracies rise to around 60{\%}. However, these results remain only marginally above random chance. Taken together, our findings identify significant challenges to be solved before AI agents can automate earth observation, and suggest paths forward."
}