Daisy Munson
2026
*-PLUIE: Personalisable metric with Llm Used for Improved Evaluation
Quentin Lemesle | Leane Jourdan | Daisy Munson | Pierre Alain | Jonathan Chevelu | Arnaud Delhay | Damien Lolive
Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)
Quentin Lemesle | Leane Jourdan | Daisy Munson | Pierre Alain | Jonathan Chevelu | Arnaud Delhay | Damien Lolive
Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)
Evaluating the quality of automatically generated text often relies on LLM-as-a-judge (LLM-judge) methods. While effective, these approaches are computationally expensive and require post-processing. To address these limitations, we build upon ParaPLUIE, a perplexity-based LLM-judge metric that estimates confidence over “Yes/No” answers without generating text. We introduce *-PLUIE, task-specific prompting variants of ParaPLUIE and evaluate their alignment with human judgement. Our experiments show that personalised *-PLUIE achieves stronger correlations with human ratings while maintaining low computational cost.