A Survey of LLM-Based Applications in Programming Education: Balancing Automation and Human Oversight

Griffin Pitts, Anurata Prabha Hridi, Arun Balajiee Lekshmi Narayanan


Abstract
Novice programmers benefit from timely, personalized support that addresses individual learning gaps, yet the availability of instructors and teaching assistants is inherently limited. Large language models (LLMs) present opportunities to scale such support, though their effectiveness depends on how well technical capabilities are aligned with pedagogical goals. This survey synthesizes recent work on LLM applications in programming education across three focal areas: formative code feedback, assessment, and knowledge modeling. We identify recurring design patterns in how these tools are applied and find that interventions are most effective when educator expertise complements model output through human-in-the-loop oversight, scaffolding, and evaluation. Fully automated approaches are often constrained in capturing the pedagogical nuances of programming education, although human-in-the-loop designs and course-specific adaptation offer promising directions for future improvement. Future research should focus on improving transparency, strengthening alignment with pedagogy, and developing systems that flexibly adapt to the needs of varied learning contexts.
Anthology ID:
2025.hcinlp-1.21
Volume:
Proceedings of the Fourth Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP)
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Su Lin Blodgett, Amanda Cercas Curry, Sunipa Dev, Siyan Li, Michael Madaio, Jack Wang, Sherry Tongshuang Wu, Ziang Xiao, Diyi Yang
Venues:
HCINLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
255–262
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.hcinlp-1.21/
DOI:
Bibkey:
Cite (ACL):
Griffin Pitts, Anurata Prabha Hridi, and Arun Balajiee Lekshmi Narayanan. 2025. A Survey of LLM-Based Applications in Programming Education: Balancing Automation and Human Oversight. In Proceedings of the Fourth Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP), pages 255–262, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
A Survey of LLM-Based Applications in Programming Education: Balancing Automation and Human Oversight (Pitts et al., HCINLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.hcinlp-1.21.pdf