@inproceedings{lemesle-etal-2026-pluie,
    title = "*-{PLUIE}: Personalisable metric with Llm Used for Improved Evaluation",
    author = "Lemesle, Quentin  and
      Jourdan, Leane  and
      Munson, Daisy  and
      Alain, Pierre  and
      Chevelu, Jonathan  and
      Delhay, Arnaud  and
      Lolive, Damien",
    editor = "Mohammad, Saif M.  and
      Ousidhoum, Nedjma",
    booktitle = "Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*{SEM} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.starsem-conference.14/",
    pages = "211--243",
    ISBN = "979-8-89176-413-2",
    abstract = "Evaluating the quality of automatically generated text often relies on LLM-as-a-judge (LLM-judge) methods. While effective, these approaches are computationally expensive and require post-processing. To address these limitations, we build upon ParaPLUIE, a perplexity-based LLM-judge metric that estimates confidence over ``Yes/No'' answers without generating text. We introduce *-PLUIE, task-specific prompting variants of ParaPLUIE and evaluate their alignment with human judgement. Our experiments show that personalised *-PLUIE achieves stronger correlations with human ratings while maintaining low computational cost."
}