@inproceedings{orsini-brunato-2026-steering,
    title = "Steering Pragmatic Interpretation in {LLM}s: A Diagnostic Evaluation of Few-Shot and Reasoning-Based Prompting for Indirect Speech Acts.",
    author = "Orsini, Massimiliano  and
      Brunato, Dominique",
    editor = "Egg, Markus  and
      Kordoni, Valia",
    booktitle = "Proceedings of Learning Non-Literal Expressions with Small Data @ {LREC} 2026",
    month = may,
    year = "2026",
    address = "Palma, Mallorca (Spain)",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/paragraph-normalization/2026.nonliteral-1.2/",
    doi = "10.63317/49dfcg89tqms",
    pages = "12--20",
    abstract = "Pragmatic competence poses a persistent challenge for large language models, as it requires context-dependent inference beyond literal meaning. This study examines whether few-shot prompting can reliably steer LLMs toward appropriate interpretations of indirect speech acts under small-data conditions. Focusing on Italian, we evaluate three LLMs on a small dataset that captures pragmatic ambiguity through graded plausibility judgments. We compare a zero-shot baseline with multiple few-shot prompting configurations that vary in the number and composition of demonstrations, as well as in the presence of explicit pragmatic guidance. Results show that few-shot prompting does not yield robust or monotonic improvements overall. While performance improves substantially for conventionalized indirect speech acts, gains for non-conventionalized indirect speech acts are unstable and limited. In contrast, introducing explicit pragmatic reasoning along with demonstrations through guided chain-of-thought prompting appears more promising. Overall, these findings highlight the limits of example-based steering for pragmatic inference and suggest that explicitly modeling pragmatic reasoning may be a more effective direction in small-data settings."
}