@inproceedings{machcha-etal-2025-large,
    title = "Do Large Language Models Know When Not to Answer in Medical {QA}?",
    author = "Machcha, Sravanthi  and
      Yerra, Sushrita  and
      Sultana, Sharmin  and
      Yu, Hong  and
      Yao, Zonghai",
    editor = "Noidea, Noidea",
    booktitle = "Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025)",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.uncertainlp-main.4/",
    pages = "27--35",
    ISBN = "979-8-89176-349-4",
    abstract = "Uncertainty awareness is essential for large language models (LLMs), particularly in safety-critical domains such as medicine where erroneous or hallucinatory outputs can cause harm. Yet most evaluations remain centered on accuracy, offering limited insight into model confidence and its relation to abstention. In this work, we present preliminary experiments that combine conformal prediction with abstention-augmented and perturbed variants of medical QA datasets. Our early results suggest a positive link between uncertainty estimates and abstention decisions, with this effect amplified under higher difficulty and adversarial perturbations. These findings highlight abstention as a practical handle for probing model reliability in medical QA."
}