@inproceedings{cohn-etal-2019-large,
    title = "A Large-Scale User Study of an {A}lexa {P}rize Chatbot: Effect of {TTS} Dynamism on Perceived Quality of Social Dialog",
    author = "Cohn, Michelle  and
      Chen, Chun-Yen  and
      Yu, Zhou",
    editor = "Nakamura, Satoshi  and
      Gasic, Milica  and
      Zukerman, Ingrid  and
      Skantze, Gabriel  and
      Nakano, Mikio  and
      Papangelis, Alexandros  and
      Ultes, Stefan  and
      Yoshino, Koichiro",
    booktitle = "Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue",
    month = sep,
    year = "2019",
    address = "Stockholm, Sweden",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/W19-5935/",
    doi = "10.18653/v1/W19-5935",
    pages = "293--306",
    abstract = "This study tests the effect of cognitive-emotional expression in an Alexa text-to-speech (TTS) voice on users' experience with a social dialog system. We systematically introduced emotionally expressive interjections (e.g., {\textquotedblleft}Wow!{\textquotedblright}) and filler words (e.g., {\textquotedblleft}um{\textquotedblright}, {\textquotedblleft}mhmm{\textquotedblright}) in an Amazon Alexa Prize socialbot, Gunrock. We tested whether these TTS manipulations improved users' ratings of their conversation across thousands of real user interactions (n=5,527). Results showed that interjections and fillers each improved users' holistic ratings, an improvement that further increased if the system used both manipulations. A separate perception experiment corroborated the findings from the user study, with improved social ratings for conversations including interjections; however, no positive effect was observed for fillers, suggesting that the role of the rater in the conversation{---}as active participant or external listener{---}is an important factor in assessing social dialogs."
}