XPersona: Evaluating Multilingual Personalized Chatbot

Zhaojiang Lin, Zihan Liu, Genta Indra Winata, Samuel Cahyawijaya, Andrea Madotto, Yejin Bang, Etsuko Ishii, Pascale Fung


Abstract
Personalized dialogue systems are an essential step toward better human-machine interaction. Existing personalized dialogue agents rely on properly designed conversational datasets, which are mostly monolingual (e.g., English), which greatly limits the usage of conversational agents in other languages. In this paper, we propose a multi-lingual extension of Persona-Chat, namely XPersona. Our dataset includes persona conversations in six different languages other than English for evaluating multilingual personalized agents. We experiment with both multilingual and cross-lingual trained baselines and evaluate them against monolingual and translation-pipeline models using both automatic and human evaluation. Experimental results show that the multilingual trained models outperform the translation pipeline and that they are on par with the monolingual models, with the advantage of having a single model across multiple languages. On the other hand, the state-of-the-art cross-lingual trained models achieve inferior performance to the other models, showing that cross-lingual conversation modeling is a challenging task. We hope that our dataset and baselines will accelerate research in multilingual dialogue systems.
Anthology ID:
2021.nlp4convai-1.10
Volume:
Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI
Month:
November
Year:
2021
Address:
Online
Venue:
NLP4ConvAI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
102–112
Language:
URL:
https://aclanthology.org/2021.nlp4convai-1.10
DOI:
10.18653/v1/2021.nlp4convai-1.10
Bibkey:
Cite (ACL):
Zhaojiang Lin, Zihan Liu, Genta Indra Winata, Samuel Cahyawijaya, Andrea Madotto, Yejin Bang, Etsuko Ishii, and Pascale Fung. 2021. XPersona: Evaluating Multilingual Personalized Chatbot. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI, pages 102–112, Online. Association for Computational Linguistics.
Cite (Informal):
XPersona: Evaluating Multilingual Personalized Chatbot (Lin et al., NLP4ConvAI 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.nlp4convai-1.10.pdf
Software:
 2021.nlp4convai-1.10.Software.zip
Video:
 https://preview.aclanthology.org/ingestion-script-update/2021.nlp4convai-1.10.mp4
Code
 HLTCHKUST/Xpersona
Data
ConvAI2