BRACU_CL at BLP-2025 Task 2: CodeMist: A Transformer-Based Framework for Bangla Instruction-to-Code Generation
Md. Fahmid-Ul-Alam Juboraj, Soumik Deb Niloy, Mahbub E Sobhani, Farig Sadeque
Abstract
This study proposes a hybrid framework for Bangla-to-Python code generation, emphasizing improved code accuracy through a two-phase pipeline: generation and debugging. During development, standalone models such as TigerLLM and StarCoder achieved modest accuracies of 27% and 24%, respectively, while more advanced models, Gemini-1.5-flash and Gemma, reached 60% and 64%. Integrating Gemma with the gpt-oss debugger substantially increased accuracy to 99.75%, highlighting the critical role of a dedicated debugging stage. In testing on unseen data, gpt-oss alone achieved 67%, which improved to 71% with self-debugging. The highest performance, 84%, was obtained by pairing Gemini-2.5-flash as the generator with gpt-oss for debugging. These findings demonstrate that combining a strong generative model with an effective debugging component yields superior and robust code generation results, outperforming existing approaches such as TigerLLM. The full implementation of the framework is publicly available at https://github.com/fahmid-juboraj/Code_generation.- Anthology ID:
- 2025.banglalp-1.67
- Volume:
- Proceedings of the Second Workshop on Bangla Language Processing (BLP-2025)
- Month:
- December
- Year:
- 2025
- Address:
- Mumbai, India
- Editors:
- Firoj Alam, Sudipta Kar, Shammur Absar Chowdhury, Naeemul Hassan, Enamul Hoque Prince, Mohiuddin Tasnim, Md Rashad Al Hasan Rony, Md Tahmid Rahman Rahman
- Venues:
- BanglaLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 656–662
- Language:
- URL:
- https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.banglalp-1.67/
- DOI:
- Cite (ACL):
- Md. Fahmid-Ul-Alam Juboraj, Soumik Deb Niloy, Mahbub E Sobhani, and Farig Sadeque. 2025. BRACU_CL at BLP-2025 Task 2: CodeMist: A Transformer-Based Framework for Bangla Instruction-to-Code Generation. In Proceedings of the Second Workshop on Bangla Language Processing (BLP-2025), pages 656–662, Mumbai, India. Association for Computational Linguistics.
- Cite (Informal):
- BRACU_CL at BLP-2025 Task 2: CodeMist: A Transformer-Based Framework for Bangla Instruction-to-Code Generation (Juboraj et al., BanglaLP 2025)
- PDF:
- https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.banglalp-1.67.pdf