BlackboxNLP-2025 MIB Shared Task: Improving Circuit Faithfulness via Better Edge Selection
Yaniv Nikankin, Dana Arad, Itay Itzhak, Anja Reusch, Adi Simhi, Gal Kesten, Yonatan Belinkov
Abstract
One of the main challenges in mechanistic interpretability is circuit discovery – determining which parts of a model perform a given task. We build on the Mechanistic Interpretability Benchmark (MIB) and propose three key improvements to circuit discovery. First, we use bootstrapping to identify edges with consistent attribution scores. Second, we introduce a simple ratio-based selection strategy to prioritize strong positive-scoring edges, balancing performance and faithfulness. Third, we replace the standard greedy selection with an integer linear programming formulation. Our methods yield more faithful circuits and outperform prior approaches across multiple MIB tasks and models.- Anthology ID:
- 2025.blackboxnlp-1.29
- Volume:
- Proceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
- Month:
- November
- Year:
- 2025
- Address:
- Suzhou, China
- Editors:
- Yonatan Belinkov, Aaron Mueller, Najoung Kim, Hosein Mohebbi, Hanjie Chen, Dana Arad, Gabriele Sarti
- Venues:
- BlackboxNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 521–527
- Language:
- URL:
- https://preview.aclanthology.org/ingest-emnlp/2025.blackboxnlp-1.29/
- DOI:
- Cite (ACL):
- Yaniv Nikankin, Dana Arad, Itay Itzhak, Anja Reusch, Adi Simhi, Gal Kesten, and Yonatan Belinkov. 2025. BlackboxNLP-2025 MIB Shared Task: Improving Circuit Faithfulness via Better Edge Selection. In Proceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 521–527, Suzhou, China. Association for Computational Linguistics.
- Cite (Informal):
- BlackboxNLP-2025 MIB Shared Task: Improving Circuit Faithfulness via Better Edge Selection (Nikankin et al., BlackboxNLP 2025)
- PDF:
- https://preview.aclanthology.org/ingest-emnlp/2025.blackboxnlp-1.29.pdf