Empowering AAC Users: A Systematic Integration of Personal Narratives with Conversational AI
Sayantan Pal, Souvik Das, Rohini Srihari, Jeff Higginborham, Jenna Bizovi
Abstract
Communication barriers have long posed challenges for users of Alternate and Augmentative Communication (AAC). In AAC, effective conversational aids are not solely about harnessing Artificial Intelligence (AI) capabilities but more about ensuring these technologies resonate deeply with AAC user’s unique communication challenges. We aim to bridge the gap between generic outputs and genuine human interactions by integrating advanced Conversational AI with personal narratives. While existing solutions offer generic responses, a considerable gap in tailoring outputs reflecting an AAC user’s intent must be addressed. Thus, we propose to create a custom conversational dataset centered on the experiences and words of a primary AAC user to fine-tune advanced language models. Additionally, we employ a Retrieval-Augmented Generation (RAG) method, drawing context from a summarized version of authored content by the AAC user. This combination ensures that responses are contextually relevant and deeply personal. Preliminary evaluations underscore its transformative potential, with automated metrics and human assessments showcasing significantly enhanced response quality.- Anthology ID:
- 2024.customnlp4u-1.2
- Volume:
- Proceedings of the 1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual (CustomNLP4U)
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Sachin Kumar, Vidhisha Balachandran, Chan Young Park, Weijia Shi, Shirley Anugrah Hayati, Yulia Tsvetkov, Noah Smith, Hannaneh Hajishirzi, Dongyeop Kang, David Jurgens
- Venues:
- CustomNLP4U | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 12–25
- Language:
- URL:
- https://preview.aclanthology.org/fix-sig-urls/2024.customnlp4u-1.2/
- DOI:
- 10.18653/v1/2024.customnlp4u-1.2
- Cite (ACL):
- Sayantan Pal, Souvik Das, Rohini Srihari, Jeff Higginborham, and Jenna Bizovi. 2024. Empowering AAC Users: A Systematic Integration of Personal Narratives with Conversational AI. In Proceedings of the 1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual (CustomNLP4U), pages 12–25, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- Empowering AAC Users: A Systematic Integration of Personal Narratives with Conversational AI (Pal et al., CustomNLP4U 2024)
- PDF:
- https://preview.aclanthology.org/fix-sig-urls/2024.customnlp4u-1.2.pdf