@inproceedings{scheinberg-etal-2025-explain,
    title = "Explain-then-Process: Using Grammar Prompting to Enhance Grammatical Acceptability Judgments",
    author = "Scheinberg, Russell  and
      Agrawal, Ameeta  and
      Shore, Amber  and
      Lee, So Young",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.1015/",
    pages = "19778--19795",
    ISBN = "979-8-89176-256-5",
    abstract = "Large language models (LLMs) can explain grammatical rules, yet they often fail to apply those rules when judging sentence acceptability. We present \textit{grammar prompting}, an explain-then-process paradigm: a large LLM first produces a concise explanation of the relevant syntactic phenomenon, then that explanation is fed back as additional context to the \textit{target model} {--} either an LLM or a smaller language model (SLM) {--} before deciding which sentence of a minimal pair is grammatical. On the English BLiMP, Chinese SLING, and Russian RuBLiMP benchmarks, this simple prompt design yields substantial improvements over strong baselines across a wide range of syntactic phenomena. Feeding an LLM{'}s metalinguistic explanation back to the target model bridges the gap between \textit{knowing} a rule and \textit{using} it. On SLMs, grammar prompting alone trims the average LLM-SLM accuracy gap by 20{\%}, and when paired with chain-of-thought, by 56{\%} (13.0 pp $\to$ 5.8 pp), all at negligible cost. The lightweight, language-agnostic cue lets low-cost SLMs approach frontier-LLM performance in multilingual settings."
}