@inproceedings{abdelwahab-etal-2026-thinking,
    title = "What are They Thinking? Delineation, Probing, and Tracking of Concepts in {LLM}s",
    author = "Abdelwahab, Mohamed  and
      Collins, Michelle Yu  and
      Chen, Sihan  and
      Zhao, Yi Cheng  and
      Mahmood, Zafarullah  and
      Zhu, Jiading  and
      Ali, Soliman  and
      Rose, Jonathan",
    editor = "Chang, Kai-Wei  and
      Mehrabi, Ninareh  and
      Krishna, Satyapriya  and
      Das, Anubrata  and
      Dhamala, Jwala  and
      Cao, Yang Trista  and
      Kumarage, Tharindu  and
      Ramakrishna, Anil  and
      Christodoulopoulos, Christos  and
      Wan, Yixin  and
      Galystan, Aram  and
      Kumar, Anoop  and
      Gupta, Rahul",
    booktitle = "Proceedings of the 6th Workshop on Trustworthy {NLP} ({T}rust{NLP} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.trustnlp-main.9/",
    pages = "121--179",
    ISBN = "979-8-89176-418-7",
    abstract = "As the influence of LLMs expands, it is imperative to gain insight into their decisions. One way to do that is to develop probes that detect the presence or absence of a broad set of high-level abstract concepts within the embeddings computed in an LLM - which is what we might say a model is ``thinking'' about. Such probes should be low-cost and easily applicable to any LLM, so that monitoring for many concepts is possible during normal operation.In this paper, we take the first steps towards developing the capability of creating many such probes by defining and executing examples of the key tasks needed: first, the careful delineation of a high-level abstract concept through the creation of a dataset with the concept both present and then absent. Then, the training and testing of a set of linear probes to detect the concept on any layer of an LLM, including an exploration of the complexity of the probe needed. Finally, we show that such probes can track concepts across larger contexts. This is done with four separate concepts and three different LLMs. When this process is scaled to many more concepts, it will create the ability to monitor new models."
}