@inproceedings{lee-etal-2026-oasis,
    title = "{OASIS}: Online Sample Selection for Continual Instruction Tuning",
    author = "Lee, Minjae  and
      Seo, Minhyuk  and
      Qu, Tingyu  and
      Tuytelaars, Tinne  and
      Choi, Jonghyun",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.158/",
    pages = "3491--3515",
    ISBN = "979-8-89176-390-6",
    abstract = "In continual instruction tuning (CIT) scenarios, where new instruction tuning data continuously arrive in an online streaming manner, training delays from large-scale data significantly hinder real-time adaptation. Data selection can mitigate this overhead, but existing strategies often rely on pre-trained reference models, which are impractical in CIT setups since future data are unknown. Recent reference model-free online sample selection methods address this, but typically select a fixed number of samples per batch (e.g., top-k), making them vulnerable to distribution shifts where informativeness varies across batches. To address these limitations, we propose OASIS, an adaptive online sample selection approach for CIT that (1) selects informative samples by estimating each sample{'}s informativeness relative to all previously seen data, beyond batch-level constraints, and (2) minimizes informative redundancy of selected samples through iterative selection score updates. Experiments on various large foundation models show that , using only 25{\%} of the data, achieves comparable performance to full-data training and outperforms the state-of-the-art sampling methods."
}