@inproceedings{mendonca-etal-2023-dialogue,
    title = "Dialogue Quality and Emotion Annotations for Customer Support Conversations",
    author = "Mendonca, John  and
      Pereira, Patr{\'\i}cia  and
      Menezes, Miguel  and
      Cabarr{\~a}o, Vera  and
      Farinha, Ana C  and
      Moniz, Helena  and
      Lavie, Alon  and
      Trancoso, Isabel",
    editor = "Gehrmann, Sebastian  and
      Wang, Alex  and
      Sedoc, Jo{\~a}o  and
      Clark, Elizabeth  and
      Dhole, Kaustubh  and
      Chandu, Khyathi Raghavi  and
      Santus, Enrico  and
      Sedghamiz, Hooman",
    booktitle = "Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.gem-1.2",
    pages = "9--21",
    abstract = "Task-oriented conversational datasets often lack topic variability and linguistic diversity. However, with the advent of Large Language Models (LLMs) pretrained on extensive, multilingual and diverse text data, these limitations seem overcome. Nevertheless, their generalisability to different languages and domains in dialogue applications remains uncertain without benchmarking datasets. This paper presents a holistic annotation approach for emotion and conversational quality in the context of bilingual customer support conversations. By performing annotations that take into consideration the complete instances that compose a conversation, one can form a broader perspective of the dialogue as a whole. Furthermore, it provides a unique and valuable resource for the development of text classification models. To this end, we present benchmarks for Emotion Recognition and Dialogue Quality Estimation and show that further research is needed to leverage these models in a production setting.",
}