Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics

Khang Tran, Khoa Nguyen, Cristian Borcea, Hai Phan


Abstract
Recent advances in large language models for test case generation have improved branch coverage via prompt-engineered mutations. However, they still lack principled mechanisms for steering models toward specific high-risk execution branches, limiting their effectiveness for discovering subtle bugs and security vulnerabilities. We propose GLMTest, the first program structure-aware LLM framework for targeted test case generation that seamlessly integrates code property graphs and code semantics using a graph neural network and a language model to condition test case generation on execution branches. This structured conditioning enables controllable and branch-targeted test case generation, thereby potentially enhancing bug and security risk discovery. Experiments on real-world projects show that GLMTest built on a Qwen2.5-Coder-7B-Instruct model improves branch accuracy from 27.4% to 50.2% on TestGenEval benchmark compared with state-of-the-art LLMs, i.e., Claude-Sonnet-4.5 and GPT-4o-mini.
Anthology ID:
2026.findings-acl.540
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11113–11126
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.findings-acl.540/
DOI:
Bibkey:
Cite (ACL):
Khang Tran, Khoa Nguyen, Cristian Borcea, and Hai Phan. 2026. Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics. In Findings of the Association for Computational Linguistics: ACL 2026, pages 11113–11126, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics (Tran et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.findings-acl.540.pdf
Checklist:
 2026.findings-acl.540.checklist.pdf