Hui Li Tan
2025
SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning
Zhengyuan Liu
|
Geyu Lin
|
Hui Li Tan
|
Huayun Zhang
|
Yanfeng Lu
|
Xiaoxue Gao
|
Stella Xin Yin
|
Sun He
|
Hock Huan Goh
|
Lung Hsiang Wong
|
Nancy F. Chen
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)
The integration of generative artificial intelligence into educational applications has enhanced personalized and interactive learning experiences, and it shows strong potential to promote young learners language acquisition. However, it is still challenging to ensure consistent and robust performance across different languages and cultural contexts, and kids-friendly design requires simplified instructions, engaging interactions, and age-appropriate scaffolding to maintain motivation and optimize learning outcomes.In this work, we introduce SingaKids, a dialogic tutor designed to facilitate language learning through picture description tasks. Our system integrates dense image captioning, multilingual dialogic interaction, speech understanding, and engaging speech generation to create an immersive learning environment in four languages: English, Mandarin, Malay, and Tamil. We further improve the system through multilingual pre-training, task-specific tuning, and scaffolding optimization. Empirical studies with elementary school students demonstrate that SingaKids provides effective dialogic teaching, benefiting learners at different performance levels.
Search
Fix author
Co-authors
- Nancy Chen 1
- Xiaoxue Gao 1
- Hock Huan Goh 1
- Sun He 1
- Geyu Lin 1
- show all...
Venues
- acl1