@inproceedings{arai-etal-2025-evaluating,
    title = "Evaluating {LLM}s' Ability to Understand Numerical Time Series for Text Generation",
    author = "Arai, Mizuki  and
      Ishigaki, Tatsuya  and
      Kawarada, Masayuki  and
      Miyao, Yusuke  and
      Takamura, Hiroya  and
      Kobayashi, Ichiro",
    editor = "Flek, Lucie  and
      Narayan, Shashi  and
      Phương, L{\^e} Hồng  and
      Pei, Jiahuan",
    booktitle = "Proceedings of the 18th International Natural Language Generation Conference",
    month = oct,
    year = "2025",
    address = "Hanoi, Vietnam",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-you-zhang-rochester/2025.inlg-main.16/",
    pages = "232--248",
    abstract = "Data-to-text generation tasks often involve processing numerical time-series as input such as financial statistics or meteorological data. Although large language models (LLMs) are a powerful approach to data-to-text, we still lack a comprehensive understanding of how well they actually understand time-series data. We therefore introduce a benchmark with 18 evaluation tasks to assess LLMs' abilities of interpreting numerical time-series, which are categorized into: 1) event detection{---}identifying maxima and minima; 2) computation{---}averaging and summation; 3) pairwise comparison{---}comparing values over time; and 4) inference{---}imputation and forecasting. Our experiments reveal five key findings: 1) even state-of-the-art LLMs struggle with complex multi-step reasoning; 2) tasks that require extracting values or performing computations within a specified range of the time-series significantly reduce accuracy; 3) instruction tuning offers inconsistent improvements for numerical interpretation; 4) reasoning-based models outperform standard LLMs in complex numerical tasks; and 5) LLMs perform interpolation better than forecasting. These results establish a clear baseline and serve as a wake-up call for anyone aiming to blend fluent language with trustworthy numeric precision in time-series scenarios."
}