Julia Farganus
2026
Factual State Discovery Benchmark: Evaluating Fact Elicitation in Polish Tax Law
Mateusz Bystroński | Kamil Tagowski | Denis Janiak | Julia Farganus | Lukasz Augustyniak | Monika Kajdanowicz | Tomasz Jan Kajdanowicz
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Mateusz Bystroński | Kamil Tagowski | Denis Janiak | Julia Farganus | Lukasz Augustyniak | Monika Kajdanowicz | Tomasz Jan Kajdanowicz
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Before a tax authority can issue a ruling, it must receive a complete description of the taxpayer’s situation—yet no benchmark measures whether language models can systematically elicit all relevant facts through dialogue.We introduce FSDBench (Factual State Discovery Benchmark), in which a discovery agent questions a simulated taxpayer grounded in a real tax document.The dataset comprises 500 narratives from official Polish tax interpretations, decomposed into 32 874 atomic facts with validated supported precision (97.6%), atomicity (93.8%), and sentence coverage (96.0%).Experiments with four models show that even the best system recovers only 77% of facts on easy samples and under 49% on hard samples after 50 turns.These findings establish conversational fact elicitation as a challenging open problem requiring retrieval-augmented and adaptive questioning strategies.