@inproceedings{fei-etal-2025-advancing,
    title = "Advancing Sequential Numerical Prediction in Autoregressive Models",
    author = "Fei, Xiang  and
      Lu, Jinghui  and
      Sun, Qi  and
      Feng, Hao  and
      Wang, Yanjie  and
      Shi, Wei  and
      Wang, An-Lan  and
      Tang, Jingqun  and
      Huang, Can",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-short.44/",
    pages = "562--574",
    ISBN = "979-8-89176-252-7",
    abstract = "Autoregressive models have become the de facto choice for sequence generation tasks, but standard approaches treat digits as independent tokens and apply cross-entropy loss, overlooking the coherent structure of numerical sequences. This paper introduces $\textit{\textbf{N}umerical \textbf{T}oken \textbf{I}ntegrity \textbf{Loss} (NTIL)}$ to address this gap. NTIL operates at two levels: (1) token-level, where it extends the Earth Mover{'}s Distance (EMD) to preserve ordinal relationships between numerical values, and (2) sequence-level, where it penalizes the overall discrepancy between the predicted and actual sequences. This dual approach improves numerical prediction and integrates effectively with LLMs/MLLMs. Extensive experiments show significant performance improvements with NTIL."
}