@inproceedings{onoeva-2026-polar,
    title = "Not All Polar Questions Are the Same: {ASR}, Humans, and {R}ussian",
    author = "Onoeva, Maria",
    editor = "Hosseini-Kivanani, Nina  and
      Brutti, Alessio  and
      Matassoni, Marco  and
      Dowerah, Sandipana  and
      Liga, Davide  and
      Schommer, Christoph",
    booktitle = "Proceedings of Speech Language Models in Low-Resource Settings: Performance, Evaluation, and Bias Analysis ({SPEAKABLE}) @ {LREC} 2026",
    month = may,
    year = "2026",
    address = "Palma, Mallorca (Spain)",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/revision-previews/2026.speakable-1.4/",
    doi = "10.63317/3qkbn7arv9es",
    pages = "31--38",
    abstract = "Word Error Rate (WER) remains the standard metric in automatic speech recognition (ASR) evaluation, yet it does not capture higher-level linguistic distinctions such as prosody. This article examines how three state-of-the-art open-source ASR models (Whisper, Meta{'}s MMS, and GigaAM) handle the distinction between Russian polar questions and assertions. Russian is particularly suitable for this investigation because polar questions can be marked either morphologically (li, razve) or purely intonationally, without changes in word order. Using audio stimuli from a controlled psycholinguistic experiment, I compare human classification performance in two experimental studies with ASR transcriptions, taking sentence-final punctuation as a proxy for prosodic interpretation. While human participants show near-ceiling accuracy, the ASR models perform inconsistently, especially on intonationally marked questions. Additional contextual cues improve performance in some cases but also reveal instability across conditions. The results demonstrate that evaluating punctuation provides insights beyond WER and allows a more fine-grained view of how current ASR systems encode prosodic and grammatical information."
}