@inproceedings{yin-etal-2024-enhancing,
    title = "Enhancing {LLM} Capabilities Beyond Scaling Up",
    author = "Yin, Wenpeng  and
      Chen, Muhao  and
      Zhang, Rui  and
      Zhou, Ben  and
      Wang, Fei  and
      Roth, Dan",
    editor = "Li, Jessy  and
      Liu, Fei",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Ingest-2025-COMPUTEL/2024.emnlp-tutorials.1/",
    doi = "10.18653/v1/2024.emnlp-tutorials.1",
    pages = "1--10",
    abstract = "General-purpose large language models (LLMs) are progressively expanding both in scale and access to unpublic training data. This has led to notable progress in a variety of AI problems. Nevertheless, two questions exist: i) Is scaling up the sole avenue of extending the capabilities of LLMs? ii) Instead of developing general-purpose LLMs, how to endow LLMs with specific knowledge? This tutorial targets researchers and practitioners who are interested in capability extension of LLMs that go beyond scaling up. To this end, we will discuss several lines of research that follow that direction, including (i) the adaptation of LLMs to assimilate new information in situations where conflicts arise, (ii) the adaptation of LLMs to address target problems with inherent constraints, (iii) the customization of LLMs to align with user-specific instructions and preference, (iv) the defense against potential attacks and threads by malicious users, and (v) the collaboration with external models directly or through APIs. At last, we will conclude the tutorial by outlining directions for further investigation."
}