@inproceedings{santos-2026-speech,
    title = "Speech Disfluencies and {LLM} Confidence: Length Bias and Pragmatic Insensitivity in {B}razilian {P}ortuguese",
    author = "Santos, Valeria",
    editor = "Braud, Chlo{\'e}  and
      Hardmeier, Christian  and
      Ogrodniczuk, Maciej  and
      Loaiciga, Sharid  and
      Zeldes, Amir  and
      Nov{\'a}k, Michal  and
      Li, Chuyuan  and
      Strube, Michael  and
      Li, Junyi Jessy",
    booktitle = "Proceedings of the 2nd Joint Workshop on Computational Approaches to Discourse, Context and Document-Level Inferences and Computational Models of Reference, Anaphora and Coreference ({CODI}-{CRAC} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.codi-1.5/",
    pages = "24--28",
    ISBN = "979-8-89176-400-2",
    abstract = "Training Large Language Models (LLMs) relies predominantly on written, curated corpora, which may limit their reliability on spontaneous speech. Oral language exhibits real-time planning markers {---} filled pauses, repetitions, false starts, and vowel lengthenings {---} that modulate epistemic commitment. This pilot study investigates how such disfluencies affect the alignment between LLM confidence and a discourse-pragmatic uncertainty proxy in a Portuguese model (Llama-3.1-8B-Instruct). Using a benchmark of 344 turns from the Roda Viva corpus, we contrast faithful Conversation Analysis transcriptions with sanitized versions and combine binned divergence metrics (ECE, OE) with rank correlation and multivariate regression analyses. We find that model confidence is overwhelmingly driven by a surface feature {---} turn length ({\$}{\{}{\textbackslash}beta{\_}{\{}{\textbackslash}text{\{}std{\}}{\}}{\}} = +14.47, p 0.001{\$}) {---} rather than by pragmatic markers of uncertainty ({\$}{\{}{\textbackslash}beta{\_}{\{}{\textbackslash}text{\{}oral{\}}{\}}{\}} = -3.09, {\{}{\textbackslash}beta{\_}{\{}{\textbackslash}text{\{}hedges{\}}{\}}{\}} = -0.97{\$}, both non-significant; {\$}R2 = 0.29{\$}). After controlling for length, residual effects of disfluency markers align in the human-expected direction but are dwarfed by length bias. We argue that this surface-feature dominance subsumes the pragmatic blindness phenomenon and explains the substantial divergence observed via ECE (41.95) and OE (4.29) between faithful and sanitized conditions."
}