@inproceedings{lin-etal-2025-judge,
    title = "Do Before You Judge: Self-Reference as a Pathway to Better {LLM} Evaluation",
    author = "Lin, Wei-Hsiang  and
      Wei, Sheng-Lun  and
      Huang, Hen-Hsen  and
      Chen, Hsin-Hsi",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1342/",
    doi = "10.18653/v1/2025.findings-emnlp.1342",
    pages = "24651--24672",
    ISBN = "979-8-89176-335-7",
    abstract = "LLM-as-Judge frameworks are increasingly popular for AI evaluation, yet research findings on the relationship between models' generation and judgment abilities remain inconsistent. We investigate this relationship through systematic dataset- and instance-level analyses across 11 models and 21 diverse tasks. Despite both capabilities relying on the same underlying knowledge, our analyses reveal they are only weakly correlated, primarily due to LLMs' sensitivity to the responses being judged. To address this, we propose a self-reference-guided evaluation strategy that leverages a model{'}s own answers as references. This approach significantly strengthens the correlation between generation and judgment abilities, offering a practical path to align these skills and providing a reliable proxy for model selection in evaluation tasks."
}