@inproceedings{maitra-etal-2025-exploring,
    title = "Exploring Gender Differences in Emoji Usage: Implications for Human-Computer Interaction",
    author = "Maitra, Arunima  and
      French, Dorothea  and
      von der Wense, Katharina",
    editor = "Blodgett, Su Lin  and
      Curry, Amanda Cercas  and
      Dev, Sunipa  and
      Li, Siyan  and
      Madaio, Michael  and
      Wang, Jack  and
      Wu, Sherry Tongshuang  and
      Xiao, Ziang  and
      Yang, Diyi",
    booktitle = "Proceedings of the Fourth Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP)",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.hcinlp-1.25/",
    pages = "317--325",
    ISBN = "979-8-89176-353-1",
    abstract = "Large language models (LLMs) have revolutionized natural language generation across various applications. Although LLMs are highly capable in many domains, they sometimes produce responses that lack coherence or fail to align with conversational norms such as turn-taking, or providing relevant acknowledgments. Conversational LLMs are widely used, but evaluation often misses pragmatic aspects of dialogue. In this paper, we evaluate how LLM-generated dialogue compares to human conversation through the lens of dialogue acts, the functional building blocks of interaction. Using the Switchboard Dialogue Act (SwDA) corpus, we prompt two widely used open-source models, Llama 2 and Mistral, to generate responses under varying context lengths. We then automatically annotate the dialogue acts of both model and human responses with a BERT classifier and compare their distributions. Our experimental findings reveal that the distribution of dialogue acts generated by these models differs significantly from the distribution of dialogue acts in human conversation, indicating an area for improvement. Perplexity analysis further highlights that certain dialogue acts like Acknowledge (Backchannel) are harder for models to predict. While preliminary, this study demonstrates the value of dialogue act analysis as a diagnostic tool for human-LLM interaction, highlighting both current limitations and directions for improvement."
}