NLP-CEIA-UFG at SemEval-2026 Task 8: Iterative Retrieval with Notes-Guided Query Refinement for Multi-Turn RAG
Guilherme Dutra, André Felipe Caraíba, Nádia Félix Da Silva, Paulo Dos Santos, Deborah Silva Fernandes, Sávio Salvarino De Oliveira
Abstract
We describe NLP-CEIA-UFG, our system forSemEval-2026 Task 8, which evaluates multi-turn retrieval-augmented generation (RAG)over heterogeneous document corpora. Ourpipeline centers on a three-iteration dynamicretrieval loop in which two gpt-oss-120b-powered modules—an Iterative Query Genera-tor and a Notes Builder—interact at each stepto diversify queries and accumulate structurednotes on information gaps. After the loop, anAnswerability Classifier routes the query to oneof three generation paths (Complete Answer,Partial Answer, or Clarification Request). Hy-brid BM25 and dense retrieval is fused via Re-ciprocal Rank Fusion and refined by the Jinalistwise reranker. The retrieval pipeline is com-piled under DSPy and optimized with GEPA.We achieve nDCG@5 of 0.4502 (rank 17/38,Subtask A) and HM = 0.3774 (rank 24/29, Sub-task C). Post-hoc analysis identifies an over-conservative Answerability Classifier as theprimary bottleneck: 75.5% of all responseswere flagged as IDK by the evaluator, includ-ing 69.8% of ANSWERABLE questions, whilethe retrieval and generation components per-form well when the classifier routes correctly.Our code is available at https://github.com/GuiiCorreia/SemEval-2026.- Anthology ID:
- 2026.semeval-1.349
- Volume:
- Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, USA
- Editors:
- Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
- Venues:
- SemEval | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2771–2778
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.349/
- DOI:
- Cite (ACL):
- Guilherme Dutra, André Felipe Caraíba, Nádia Félix Da Silva, Paulo Dos Santos, Deborah Silva Fernandes, and Sávio Salvarino De Oliveira. 2026. NLP-CEIA-UFG at SemEval-2026 Task 8: Iterative Retrieval with Notes-Guided Query Refinement for Multi-Turn RAG. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 2771–2778, San Diego, California, USA. Association for Computational Linguistics.
- Cite (Informal):
- NLP-CEIA-UFG at SemEval-2026 Task 8: Iterative Retrieval with Notes-Guided Query Refinement for Multi-Turn RAG (Dutra et al., SemEval 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.349.pdf