@inproceedings{an-etal-2025-lcirc,
    title = "{LCIRC}: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in {LLM}s",
    author = "An, Sumin  and
      Sung, Junyoung  and
      Park, Wonpyo  and
      Park, Chanjun  and
      Seo, Paul Hongsuck",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Ingest-2025-COMPUTEL/2025.naacl-long.524/",
    pages = "10431--10442",
    ISBN = "979-8-89176-189-6",
    abstract = "While large language models (LLMs) excel in generating coherent and contextually rich outputs, their capacity to efficiently handle long-form contexts is limited by fixed-length position embeddings. Additionally, the computational cost of processing long sequences increases quadratically, making it challenging to extend context length. To address these challenges, we propose Long-form Context Injection with Recurrent Compression (LCIRC), a method that enables the efficient processing long-form sequences beyond the model`s length limit through recurrent compression without retraining the entire model. We further introduce query dependent context modeling, which selectively compresses query-relevant information, ensuring that the model retains the most pertinent content. Our empirical results demonstrate that Query Dependent LCIRC (QD-LCIRC) significantly improves LLM`s ability to manage extended contexts, making it well-suited for tasks that require both comprehensive context understanding and query relevance."
}