@inproceedings{yu-etal-2025-beyond-pointwise,
    title = "Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of {LLM} Responses",
    author = "Yu, Fangyi  and
      Seedat, Nabeel  and
      Herrmannova, Drahomira  and
      Schilder, Frank  and
      Schwarz, Jonathan Richard",
    editor = "Potdar, Saloni  and
      Rojas-Barahona, Lina  and
      Montella, Sebastien",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track",
    month = nov,
    year = "2025",
    address = "Suzhou (China)",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-industry.136/",
    pages = "1931--1954",
    ISBN = "979-8-89176-333-3",
    abstract = "Evaluating long-form answers in high-stakes domains such as law or medicine remains a fundamental challenge. Standard metrics like BLEU and ROUGE fail to capture semantic correctness, and current LLM-based evaluators often reduce nuanced aspects of answer quality into a single undifferentiated score. We introduce DeCE, a decomposed LLM evaluation framework that separates precision (factual accuracy and relevance) and recall (coverage of required concepts), using instance-specific criteria automatically extracted from gold answer requirements. DeCE is model-agnostic and domain-general, requiring no predefined taxonomies or handcrafted rubrics. We instantiate DeCE to evaluate different LLMs on a real-world legal QA task involving multi-jurisdictional reasoning and citation grounding. DeCE achieves substantially stronger correlation with expert judgments (r=0.78), compared to traditional metrics (r=0.12) and pointwise LLM scoring (r=0.35). It also reveals interpretable trade-offs: generalist models favor recall, while specialized models favor precision. Importantly, only 11.95{\%} of LLM-generated criteria required expert revision, underscoring DeCE{'}s scalability. DeCE offers an interpretable and actionable LLM evaluation framework in expert domains."
}