@inproceedings{dumitran-etal-2025-grile,
    title = "{GRILE}: A Benchmark for Grammar Reasoning and Explanation in {R}omanian {LLM}s",
    author = "Dumitran, Marius  and
      Dumitran, Angela  and
      Danila, Alexandra Mihaela",
    editor = "Angelova, Galia  and
      Kunilovskaya, Maria  and
      Escribe, Marie  and
      Mitkov, Ruslan",
    booktitle = "Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI Era",
    month = sep,
    year = "2025",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://preview.aclanthology.org/corrections-2026-01/2025.ranlp-1.39/",
    pages = "316--324",
    abstract = "Large language models (LLMs) have revolutionised NLP, yet their pedagogical value for low{-}resource languages remains unclear. We present GRILE, the first open benchmark of 1 151 multiple{-}choice questions harvested from Romanian high{-}stakes exams (National Evaluation, Baccalaureate, university admissions). GRILE enables us to probe two complementary abilities of seven state{-}of{-}the{-}art multilingual and Romanian{-}specific LLMs: (i) selecting the correct answer, and (ii) producing linguistically faithful explanations. While Gemini 2{\textperiodcentered}5 Pro reaches 83{\%} accuracy, most open{-}weight models stay below 65{\%}, and 48{\%} of their explanations contain factual or pedagogical flaws according to expert review. A detailed error analysis pinpoints systematic weaknesses in morphology and in applying the latest DOOM 3 orthographic norms. All data, code and a public web demo are released to catalyse future research. Our findings expose open challenges for trustworthy educational NLP in low{-}resource settings and establish GRILE as a new test{-}bed for controllable explanation generation and evaluation."
}