@inproceedings{lee-etal-2026-instruction,
    title = "Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact",
    author = "Lee, Hyunji  and
      Yoon, Seunghyun  and
      Won, Yunjae  and
      Oh, Hanseok  and
      Kim, Geewook  and
      Bui, Trung  and
      Dernoncourt, Franck  and
      Stengel-Eskin, Elias  and
      Bansal, Mohit  and
      Seo, Minjoon",
    editor = "Demberg, Vera  and
      Inui, Kentaro  and
      Marquez, Llu{\'i}s",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.320/",
    pages = "6790--6810",
    ISBN = "979-8-89176-380-7",
    abstract = "Instruction tuning is a widely used approach to improve the instruction-following ability of large language models (LLMs). Instruction-tuning datasets typically include a mixture of context-augmented and context-free examples, yet prior work has largely combined these data types without examining their distinct effects. In this paper, we investigate how training LLMs with or without context affects model behavior and downstream performance. First, in the text domain, we show that LLMs trained with context attend more strongly to the provided knowledge, achieving better grounding. We also observe that context-augmented training shifts how LLMs use knowledge: models store and leverage less on parametric knowledge and instead depend more on the provided context. Second, we observe that using LLM trained with context-augmented data as the backbone for vision-language models reduces hallucination and improves grounding in the visual domain. Finally, we explore practical strategies for real-world deployments where context availability varies. We show that maintaining separate context-augmented and context-free models and routing inputs between them yields more robust overall performance than training a single mixed model, as it better preserves their complementary strengths."
}