@inproceedings{ringel-etal-2025-learning,
    title = "Learning a Continue-Thinking Token for Enhanced Test-Time Scaling",
    author = "Ringel, Liran  and
      Tolochinsky, Elad  and
      Romano, Yaniv",
    editor = "Inui, Kentaro  and
      Sakti, Sakriani  and
      Wang, Haofen  and
      Wong, Derek F.  and
      Bhattacharyya, Pushpak  and
      Banerjee, Biplab  and
      Ekbal, Asif  and
      Chakraborty, Tanmoy  and
      Singh, Dhirendra Pratap",
    booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.ijcnlp-long.177/",
    pages = "3324--3345",
    ISBN = "979-8-89176-298-5",
    abstract = "Test-time scaling has emerged as an effective approach for improving language model performance by utilizing additional compute at inference time. Recent studies have shown that overriding end-of-thinking tokens (e.g., replacing ``/think{\ensuremath{>}}'' with ``Wait'') can extend reasoning steps and improve accuracy. In this work, we explore whether a dedicated continue-thinking token can be learned to trigger extended reasoning. We augment distilled versions of DeepSeek-R1 with a single learned ``{\ensuremath{<}}|continue-thinking|{\ensuremath{>}}'' token, training only its embedding via reinforcement learning while keeping the model weights frozen. Our experiments show that this learned token achieves improved accuracy on standard math benchmarks compared to both the baseline model and a test-time scaling approach that uses a fixed token (e.g., ``Wait'') for budget forcing. In particular, we observe that in cases where the fixed-token approach enhances the base model{'}s accuracy, our method achieves a markedly greater improvement. For example, on the GSM8K benchmark, the fixed-token approach yields a 1.3{\%} absolute improvement in accuracy, whereas our learned-token method achieves a 4.2{\%} improvement over the base model that does not use budget forcing."
}