MATILDA - Multi-AnnoTator multi-language InteractiveLight-weight Dialogue Annotator

Davide Cucurnia, Nikolai Rozanov, Irene Sucameli, Augusto Ciuffoletti, Maria Simi

Abstract
Dialogue Systems are becoming ubiquitous in various forms and shapes - virtual assistants(Siri, Alexa, etc.), chat-bots, customer sup-port, chit-chat systems just to name a few.The advances in language models and their publication have democratised advanced NLP.However, data remains a crucial bottleneck.Our contribution to this essential pillar isMATILDA, to the best of our knowledge the first multi-annotator, multi-language dialogue annotation tool. MATILDA allows the creation of corpora, the management of users, the annotation of dialogues, the quick adaptation of the user interface to any language and the resolution of inter-annotator disagreement. We evaluate the tool on ease of use, annotation speed and interannotation resolution for both experts and novices and conclude that this tool not only supports the full pipeline for dialogue annotation, but also allows non-technical people to easily use it. We are completely open-sourcing the tool at https://github.com/wluper/matilda and provide a tutorial video1.
Anthology ID:
2021.eacl-demos.5
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations
Month:
April
Year:
2021
Address:
Online
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
32–39
Language:
URL:
https://aclanthology.org/2021.eacl-demos.5
DOI:
10.18653/v1/2021.eacl-demos.5
Bibkey:
Cite (ACL):
Davide Cucurnia, Nikolai Rozanov, Irene Sucameli, Augusto Ciuffoletti, and Maria Simi. 2021. MATILDA - Multi-AnnoTator multi-language InteractiveLight-weight Dialogue Annotator. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 32–39, Online. Association for Computational Linguistics.
Cite (Informal):
MATILDA - Multi-AnnoTator multi-language InteractiveLight-weight Dialogue Annotator (Cucurnia et al., EACL 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/remove-xml-comments/2021.eacl-demos.5.pdf
Code
 wluper/matilda
Data
MultiWOZ
PDF Search Code