@inproceedings{shinto-etal-2025-extent,
    title = "To What Extent Can In-Context Learning Solve Unseen Tasks?",
    author = "Shinto, Ryoma  and
      Takeshita, Masashi  and
      Rzepka, Rafal  and
      Itoh, Toshihiko",
    editor = "T.y.s.s, Santosh  and
      Shimizu, Shuichiro  and
      Gong, Yifan",
    booktitle = "The 14th International Joint Conference on Natural Language Processing and The 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.ijcnlp-srw.23/",
    pages = "277--288",
    ISBN = "979-8-89176-304-3",
    abstract = "While Large Language Models (LLMs) are known for their In-Context Learning (ICL) capabilities, there is no consensus on the underlying mechanisms. A key point of debate is whether ICL allows models to adapt to unseen tasks without parameter updates{---}that is, whether they can extrapolate. In this study, we address this question by constructing an arithmetic dataset based on the bivariate linear function $z=ax+by$ to train a model and quantitatively evaluate its interpolation and extrapolation abilities through ICL. Our results show that while extrapolation was not achieved within our experimental design, tasks that were partially learned could be solved. We also found that the model acquires internal representations that can distinguish unseen tasks, and that greater task diversity in the training dataset improves ICL capabilities."
}