@inproceedings{cheng-etal-2026-glyph,
    title = "Glyph: Scaling Context Windows via Visual-Text Compression",
    author = "Cheng, Jiale  and
      Liu, Yusen  and
      Zhang, Xinyu  and
      Fei, Yulin  and
      Hong, Wenyi  and
      Lyu, Ruiliang  and
      Wang, Weihan  and
      Su, Zhe  and
      Gu, Xiaotao  and
      Liu, Xiao  and
      Bai, Yushi  and
      Tang, Jie  and
      Wang, Hongning  and
      Huang, Minlie",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1722/",
    pages = "37145--37158",
    ISBN = "979-8-89176-390-6",
    abstract = "Large language models (LLMs) conventionally represent text as sequences of discrete tokens, making long-context scaling largely a matter of processing more tokens more efficiently.We instead explore a complementary direction: increasing how much original context each token represents.To this end, we introduce Glyph, a framework that renders long texts into compact visual pages and processes them with a vision-language model (VLM), allowing a fixed context window to cover substantially more text.To make visual compression practical, Glyph combines continual pre-training on rendered long-text data, an LLM-driven genetic search to identify rendering configurations that balance compression and task performance, and post-training with supervised fine-tuning and reinforcement learning.Across multiple long-context benchmarks, Glyph achieves 3{--}4{\texttimes} token compression while maintaining performance comparable to strong text-only LLMs such as Qwen3-8B, with over 4{\texttimes} faster prefilling and decoding and 2{\texttimes} faster supervised fine-tuning.Under more aggressive compression, a VLM with a 128K context window can handle tasks that would otherwise require up to 1M input tokens.Our code and model are released at https://github.com/thu-coai/Glyph."
}