@inproceedings{xu-etal-2026-controllable,
    title = "How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities",
    author = "Xu, Ziwen  and
      Xu, Kewei  and
      Xu, Haoming  and
      Hong, Haiwen  and
      Huang, Longtao  and
      Xue, Hui  and
      Zhang, Ningyu  and
      Shen, Yongliang  and
      Zheng, Guozhou  and
      Chen, Huajun  and
      Deng, Shumin",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1443/",
    pages = "31269--31299",
    ISBN = "979-8-89176-390-6",
    abstract = "Large Language Models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors, ranging from misaligned intent to inconsistent personality, pose significant risks. We introduce SteerEval, a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality. Each domain is structured into three specification levels: L1 (what to express), L2 (how to express), and L3 (how to instantiate), connecting high-level behavioral intent to concrete textual output. Using SteerBench, we systematically evaluate contemporary steering methods, revealing that control often degrades at finer-grained levels. Our benchmark offers a principled and interpretable framework for safe and controllable LLM behavior, serving as a foundation for future research."
}