ThinkVision at SemEval-2026 Task 6: A Transformer-Based Ensemble System for Clarity Detection
Purohit Ghanshyam, Praveen Swami, Shriyans Sahoo, Jenish Bhati, Supriya Nadiger, Sunil Saumya
Abstract
We study the problem of assessing the clarity of political question–answer pairs, where the goal is to determine whether a response directly addresses the question, avoids it, or remains ambiguous. This task is particularly challenging in political discourse, where evasiveness can be subtle and context-dependent.To tackle this problem, we propose an ensemble-based approach built on the transformer-based model DeBERTa-v3-base, fine-tuned on concatenated question–answer inputs. Special attention is given to class imbalance during training to ensure robust performance across all categories.To better capture uncertainty in borderline cases, we train multiple models with different random seeds and employ Monte Carlo Dropout at inference time. Final predictions are obtained by averaging logits across ensemble models and stochastic forward passes, yielding more stable and robust predictions.Our system achieves a Macro-F1 score of 0.76 on the evaluation dataset. Error analysis reveals that responses that partially engage with the question while failing to provide a direct answer remain the most challenging, highlighting the inherent difficulty of detecting nuanced evasiveness in political communication.- Anthology ID:
- 2026.semeval-1.248
- Volume:
- Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, USA
- Editors:
- Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
- Venues:
- SemEval | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1973–1978
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.248/
- DOI:
- Cite (ACL):
- Purohit Ghanshyam, Praveen Swami, Shriyans Sahoo, Jenish Bhati, Supriya Nadiger, and Sunil Saumya. 2026. ThinkVision at SemEval-2026 Task 6: A Transformer-Based Ensemble System for Clarity Detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 1973–1978, San Diego, California, USA. Association for Computational Linguistics.
- Cite (Informal):
- ThinkVision at SemEval-2026 Task 6: A Transformer-Based Ensemble System for Clarity Detection (Ghanshyam et al., SemEval 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.248.pdf