UNH @ Rag4Reports: A Broad Exploration of LLM-Judges for RAG

Minna Tran, Ryan McCarthy, Aiden Parsons, Jaren Unzen, Laura Dietz


Abstract
We submitted a breadth of LLM-as-a-Judge approaches to Rag4Reports Task A; our top method ranked first among all submitted systems. We find that citation faithfulness is the most essential signal, and that content is best verified by checking whether cited documents cover nuggets generated from the LLM’s internal knowledge.
Anthology ID:
2026.rag4reports-1.9
Volume:
Proceedings of the 1st Workshop on Multilingual Report Generation via Retrieval Augmented Generation (RAG4Reports 2026)
Month:
July
Year:
2026
Address:
San Diego, CA, USA
Editors:
Eugene Yang, Dawn Lawrie, Sean MacAvaney, James Mayfield, Luca Soldaini, Andrew Yates
Venues:
RAG4Reports | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
71–76
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.rag4reports-1.9/
DOI:
Bibkey:
Cite (ACL):
Minna Tran, Ryan McCarthy, Aiden Parsons, Jaren Unzen, and Laura Dietz. 2026. UNH @ Rag4Reports: A Broad Exploration of LLM-Judges for RAG. In Proceedings of the 1st Workshop on Multilingual Report Generation via Retrieval Augmented Generation (RAG4Reports 2026), pages 71–76, San Diego, CA, USA. Association for Computational Linguistics.
Cite (Informal):
UNH @ Rag4Reports: A Broad Exploration of LLM-Judges for RAG (Tran et al., RAG4Reports 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.rag4reports-1.9.pdf