Abstract
Enabling empathetic behavior in Arabic dialogue agents is an important aspect of building human-like conversational models. While Arabic Natural Language Processing has seen significant advances in Natural Language Understanding (NLU) with language models such as AraBERT, Natural Language Generation (NLG) remains a challenge. The shortcomings of NLG encoder-decoder models are primarily due to the lack of Arabic datasets suitable to train NLG models such as conversational agents. To overcome this issue, we propose a transformer-based encoder-decoder initialized with AraBERT parameters. By initializing the weights of the encoder and decoder with AraBERT pre-trained weights, our model was able to leverage knowledge transfer and boost performance in response generation. To enable empathy in our conversational model, we train it using the ArabicEmpatheticDialogues dataset and achieve high performance in empathetic response generation. Specifically, our model achieved a low perplexity value of 17.0 and an increase in 5 BLEU points compared to the previous state-of-the-art model. Also, our proposed model was rated highly by 85 human evaluators, validating its high capability in exhibiting empathy while generating relevant and fluent responses in open-domain settings.- Anthology ID:
- 2021.wanlp-1.17
- Volume:
- Proceedings of the Sixth Arabic Natural Language Processing Workshop
- Month:
- April
- Year:
- 2021
- Address:
- Kyiv, Ukraine (Virtual)
- Editors:
- Nizar Habash, Houda Bouamor, Hazem Hajj, Walid Magdy, Wajdi Zaghouani, Fethi Bougares, Nadi Tomeh, Ibrahim Abu Farha, Samia Touileb
- Venue:
- WANLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 164–172
- Language:
- URL:
- https://aclanthology.org/2021.wanlp-1.17
- DOI:
- Cite (ACL):
- Tarek Naous, Wissam Antoun, Reem Mahmoud, and Hazem Hajj. 2021. Empathetic BERT2BERT Conversational Model: Learning Arabic Language Generation with Little Data. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 164–172, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
- Cite (Informal):
- Empathetic BERT2BERT Conversational Model: Learning Arabic Language Generation with Little Data (Naous et al., WANLP 2021)
- PDF:
- https://preview.aclanthology.org/ingest-2024-clasp/2021.wanlp-1.17.pdf
- Code
- aub-mind/Arabic-Empathetic-Chatbot