@inproceedings{okur-etal-2022-end,
    title = "End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics",
    author = "Okur, Eda  and
      Sahay, Saurav  and
      Fuentes Alba, Roddy  and
      Nachman, Lama",
    editor = "Ferreira, Deborah  and
      Valentino, Marco  and
      Freitas, Andre  and
      Welleck, Sean  and
      Schubotz, Moritz",
    booktitle = "Proceedings of the 1st Workshop on Mathematical Natural Language Processing (MathNLP)",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates (Hybrid)",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2022.mathnlp-1.7/",
    doi = "10.18653/v1/2022.mathnlp-1.7",
    pages = "51--64",
    abstract = "The advances in language-based Artificial Intelligence (AI) technologies applied to build educational applications can present AI for social-good opportunities with a broader positive impact. Across many disciplines, enhancing the quality of mathematics education is crucial in building critical thinking and problem-solving skills at younger ages. Conversational AI systems have started maturing to a point where they could play a significant role in helping students learn fundamental math concepts. This work presents a task-oriented Spoken Dialogue System (SDS) built to support play-based learning of basic math concepts for early childhood education. The system has been evaluated via real-world deployments at school while the students are practicing early math concepts with multimodal interactions. We discuss our efforts to improve the SDS pipeline built for math learning, for which we explore utilizing MathBERT representations for potential enhancement to the Natural Language Understanding (NLU) module. We perform an end-to-end evaluation using real-world deployment outputs from the Automatic Speech Recognition (ASR), Intent Recognition, and Dialogue Manager (DM) components to understand how error propagation affects the overall performance in real-world scenarios."
}