The Path Not Taken: Duality in Reasoning about Program Execution

Eshgin Hasanov, Md. Mahadi Hassan, Santu Karmaker, Aashish Yadavally


Abstract
Large language models (LLMs) have shown remarkable capabilities across diverse coding tasks. However, their adoption requires a true understanding of program execution rather than relying on surface-level patterns. Existing benchmarks primarily focus on predicting program properties tied to specific inputs (e.g., code coverage, program outputs). As a result, they provide a narrow view of dynamic code reasoning and are prone to data contamination. We argue that understanding program execution requires evaluating its inherent duality through two complementary reasoning tasks: (i) predicting a program’s observed behavior for a given input, and (ii) inferring how the input must be mutated toward a specific behavioral objective. Both tasks jointly probe a model’s causal understanding of execution flow. We instantiate this duality in DexBench, a benchmark comprising 445 paired instances, and evaluate 13 LLMs. Our results demonstrate that dual-path reasoning provides a robust and discriminative proxy for dynamic code understanding.
Anthology ID:
2026.acl-long.735
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16165–16180
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.735/
DOI:
Bibkey:
Cite (ACL):
Eshgin Hasanov, Md. Mahadi Hassan, Santu Karmaker, and Aashish Yadavally. 2026. The Path Not Taken: Duality in Reasoning about Program Execution. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16165–16180, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
The Path Not Taken: Duality in Reasoning about Program Execution (Hasanov et al., ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.735.pdf
Checklist:
 2026.acl-long.735.checklist.pdf