@inproceedings{piotrowski-etal-2025-will,
    title = "When Will the Tokens End? Graph-Based Forecasting for {LLM}s Output Length",
    author = "Piotrowski, Grzegorz  and
      Bystro{\'n}ski, Mateusz  and
      Ho{\l}ysz, Miko{\l}aj  and
      Binkowski, Jakub  and
      Chodak, Grzegorz  and
      Kajdanowicz, Tomasz Jan",
    editor = "Zhao, Jin  and
      Wang, Mingyang  and
      Liu, Zhu",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.acl-srw.61/",
    pages = "843--848",
    ISBN = "979-8-89176-254-1",
    abstract = "Large Language Models (LLMs) are typically trained to predict the next token in a sequence. However, their internal representations often encode signals that go beyond immediate next-token prediction. In this work, we investigate whether these hidden states also carry information about the remaining length of the generated output{---}an implicit form of foresight (CITATION). We formulate this as a regression problem where, at generation step $t$, the target is the number of remaining tokens $y_t = T - t$, with $T$ as the total output length.We propose two approaches: (1) an aggregation-based model that combines hidden states from multiple transformer layers $\ell \in \{8, \dots, 15\}$ using element-wise operations such as mean or sum, and (2) a \textit{Layerwise Graph Regressor} that treats layerwise hidden states as nodes in a fully connected graph and applies a Graph Neural Network (GNN) to predict $y_t$. Both models operate on frozen LLM embeddings without requiring end-to-end fine-tuning.Accurately estimating remaining output length has both theoretical and practical implications. From an interpretability standpoint, it suggests that LLMs internally track their generation progress. From a systems perspective, it enables optimizations such as output-length-aware scheduling (CITATION). Our graph-based model achieves state-of-the-art performance on the Alpaca dataset using LLaMA-3-8B-Instruct, reducing normalized mean absolute error (NMAE) by over 50{\%} in short-output scenarios."
}