@inproceedings{katerenchuk-etal-2026-finrag,
    title = "{F}in{RAG}-12{B}: A Production-Validated Recipe for Grounded Question Answering in Banking",
    author = "Katerenchuk, Denys  and
      Duboue, Pablo  and
      Evanini, Keelan",
    editor = "Li, Yunyao  and
      Rehm, Georg  and
      Tu, Mei",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics ({ACL} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-industry.92/",
    pages = "1317--1328",
    ISBN = "979-8-89176-394-4",
    abstract = "Large language models (LLMs) are rapidly being adopted across various domains. However, their adoption in banking industry faces resistance due to demands for high accuracy, regulatory compliance, and the need for verifiable and grounded responses. We present a unified, data-efficient framework for training grounded domain-specific LLMs that optimizes answer quality, citation grounding, and calibrated refusal under real-world deployment constraints. First, we describe a data generation pipeline that combines LLM-as-a-Judge filtering, citation annotation, and curriculum learning with only 143M tokens. The resulting 12B model achieves high answer quality outperforming GPT-4.1 on citation grounding, with a modest citation tradeoff versus the untuned base. Second, we propose a calibrated refusal mechanism: training on 22{\%} unanswerable examples yield a 12{\%} ``I don{'}t know'' rate, substantially improving over the base model{'}s unsafe 4.3{\%} rate while avoiding GPT-4.1{'}s over-refusal (20.2{\%}). Third, we present an end-to-end methodology spanning from data curation to quantized serving. The system is deployed at 40+ financial institutions, achieving a 7.1percentage point improvement in query resolution (p {\ensuremath{<}} 0.001). Additionally, the model delivers 3{--}5x faster responses at 20{--}50x lower cost compared to GPT-4.1."
}