NLP-CEIA-UFG at SemEval-2026 Task 8: Iterative Retrieval with Notes-Guided Query Refinement for Multi-Turn RAG

Guilherme Dutra, André Felipe Caraíba, Nádia Félix Da Silva, Paulo Dos Santos, Deborah Silva Fernandes, Sávio Salvarino De Oliveira


Abstract
We describe NLP-CEIA-UFG, our system forSemEval-2026 Task 8, which evaluates multi-turn retrieval-augmented generation (RAG)over heterogeneous document corpora. Ourpipeline centers on a three-iteration dynamicretrieval loop in which two gpt-oss-120b-powered modules—an Iterative Query Genera-tor and a Notes Builder—interact at each stepto diversify queries and accumulate structurednotes on information gaps. After the loop, anAnswerability Classifier routes the query to oneof three generation paths (Complete Answer,Partial Answer, or Clarification Request). Hy-brid BM25 and dense retrieval is fused via Re-ciprocal Rank Fusion and refined by the Jinalistwise reranker. The retrieval pipeline is com-piled under DSPy and optimized with GEPA.We achieve nDCG@5 of 0.4502 (rank 17/38,Subtask A) and HM = 0.3774 (rank 24/29, Sub-task C). Post-hoc analysis identifies an over-conservative Answerability Classifier as theprimary bottleneck: 75.5% of all responseswere flagged as IDK by the evaluator, includ-ing 69.8% of ANSWERABLE questions, whilethe retrieval and generation components per-form well when the classifier routes correctly.Our code is available at https://github.com/GuiiCorreia/SemEval-2026.
Anthology ID:
2026.semeval-1.349
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2771–2778
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.349/
DOI:
Bibkey:
Cite (ACL):
Guilherme Dutra, André Felipe Caraíba, Nádia Félix Da Silva, Paulo Dos Santos, Deborah Silva Fernandes, and Sávio Salvarino De Oliveira. 2026. NLP-CEIA-UFG at SemEval-2026 Task 8: Iterative Retrieval with Notes-Guided Query Refinement for Multi-Turn RAG. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 2771–2778, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
NLP-CEIA-UFG at SemEval-2026 Task 8: Iterative Retrieval with Notes-Guided Query Refinement for Multi-Turn RAG (Dutra et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.349.pdf