UNH @ Rag4Reports: A Broad Exploration of LLM-Judges for RAG
Minna Tran, Ryan McCarthy, Aiden Parsons, Jaren Unzen, Laura Dietz
Abstract
We submitted a breadth of LLM-as-a-Judge approaches to Rag4Reports Task A; our top method ranked first among all submitted systems. We find that citation faithfulness is the most essential signal, and that content is best verified by checking whether cited documents cover nuggets generated from the LLM’s internal knowledge.- Anthology ID:
- 2026.rag4reports-1.9
- Volume:
- Proceedings of the 1st Workshop on Multilingual Report Generation via Retrieval Augmented Generation (RAG4Reports 2026)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, CA, USA
- Editors:
- Eugene Yang, Dawn Lawrie, Sean MacAvaney, James Mayfield, Luca Soldaini, Andrew Yates
- Venues:
- RAG4Reports | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 71–76
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.rag4reports-1.9/
- DOI:
- Cite (ACL):
- Minna Tran, Ryan McCarthy, Aiden Parsons, Jaren Unzen, and Laura Dietz. 2026. UNH @ Rag4Reports: A Broad Exploration of LLM-Judges for RAG. In Proceedings of the 1st Workshop on Multilingual Report Generation via Retrieval Augmented Generation (RAG4Reports 2026), pages 71–76, San Diego, CA, USA. Association for Computational Linguistics.
- Cite (Informal):
- UNH @ Rag4Reports: A Broad Exploration of LLM-Judges for RAG (Tran et al., RAG4Reports 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.rag4reports-1.9.pdf