@inproceedings{yaldiz-etal-2025-truthtorchlm,
    title = "{T}ruth{T}orch{LM}: A Comprehensive Library for Predicting Truthfulness in {LLM} Outputs",
    author = {Yaldiz, Duygu Nur  and
      Bakman, Yavuz Faruk  and
      Kang, Sungmin  and
      {\"O}zi{\c{s}}, Alperen  and
      Yildiz, Hayrettin Eren  and
      Shah, Mitash Ashish  and
      Huang, Zhiqi  and
      Kumar, Anoop  and
      Samuel, Alfy  and
      Liu, Daben  and
      Karimireddy, Sai Praneeth  and
      Avestimehr, Salman},
    editor = {Habernal, Ivan  and
      Schulam, Peter  and
      Tiedemann, J{\"o}rg},
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-demos.54/",
    pages = "717--728",
    ISBN = "979-8-89176-334-0",
    abstract = "Generative Large Language Models (LLMs) inevitably produce untruthful responses. Accurately predicting the truthfulness of these outputs is critical, especially in high-stakes settings. To accelerate research in this domain and make truthfulness prediction methods more accessible, we introduce TruthTorchLM an open-source, comprehensive Python library featuring over 30 truthfulness prediction methods, which we refer to as Truth Methods. Unlike existing toolkits such as Guardrails, which focus solely on document-grounded verification, or LM-Polygraph, which is limited to uncertainty-based methods, TruthTorchLM offers a broad and extensible collection of techniques. These methods span diverse trade-offs in computational cost, access level (e.g., black-box vs. white-box), grounding document requirements, and supervision type (self-supervised or supervised). TruthTorchLM is seamlessly compatible with both HuggingFace and LiteLLM, enabling support for locally hosted and API-based models. It also provides a unified interface for generation, evaluation, calibration, and long-form truthfulness prediction, along with a flexible framework for extending the library with new methods. We conduct an evaluation of representative truth methods on three datasets, TriviaQA, GSM8K, and FactScore-Bio."
}