@inproceedings{wang-etal-2025-recall,
    title = "{RECALL}: {RE}presentation-aligned Catastrophic-forgetting {ALL}eviation via Hierarchical Model Merging",
    author = "Wang, Bowen  and
      Wan, Haiyuan  and
      Shi, Liwen  and
      Yang, Chen  and
      He, Peng  and
      Ma, Yue  and
      Han, Haochen  and
      Li, Wenhao  and
      Tan, Tiao  and
      Li, Yongjian  and
      Liu, Fangming  and
      Yifan, Gong  and
      Zhang, Sheng",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.829/",
    pages = "16392--16406",
    ISBN = "979-8-89176-332-6",
    abstract = "We unveil that internal representations in large language models (LLMs) serve as reliable proxies of learned knowledge, and propose **RECALL**, a novel representation-aware model merging framework for continual learning without access to historical data. RECALL computes inter-model similarity from layer-wise hidden representations over clustered typical samples, and performs adaptive, hierarchical parameter fusion to align knowledge across models. This design enables the preservation of domain-general features in shallow layers while allowing task-specific adaptation in deeper layers. Unlike prior methods that require task labels or incur performance trade-offs, RECALL achieves seamless multi-domain integration and strong resistance to catastrophic forgetting. Extensive experiments across five NLP tasks and multiple continual learning scenarios show that RECALL outperforms baselines in both knowledge retention and generalization, providing a scalable and data-free solution for evolving LLMs."
}