@inproceedings{villata-etal-2026-humanica,
    title = "{H}umani{CA}: A Benchmark Resource for the Detection of Users' Ascription of Humanness to Conversational Agents",
    author = "Villata, Sabrina  and
      Rapp, Amon  and
      Di Caro, Luigi  and
      Cena, Federica",
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    booktitle = "Proceedings of the Fifteenth Language Resources and Evaluation Conference",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://preview.aclanthology.org/cawl-year/2026.lrec-1.221/",
    doi = "10.63317/3uujwof4yj3o",
    pages = "2825--2835",
    abstract = "Anthropomorphizing, which involves attributing human-like characteristics to non-human entities, is common in users' conversations with text-based conversational agents and can lead to a misalignment between the users' expectations and the agent{'}s actual capabilities. Detecting users' ascriptions of humanness automatically may enable systems to identify when users adopt a human-like style when conversing with an agent and to adapt its responses accordingly to tune their expectations. In this paper, we introduce HumaniCA, a benchmark resource comprising three annotated datasets of user turns from real dialogues with three different types of conversational agents (task-oriented, Q{\&}A, and LLM-based) aimed at indicating whether the user is ascribing humanness to the conversational agent. We also identified a set of linguistic indicators of user ascription of humanness to conversational agents and validated their utility with benchmark experiments. We then compared performance of our linguistic features and other well-known textual features (TF-IDF weights and SentenceBERT word embeddings), as well as their combinations. The evaluation highlights the central role of our linguistic features: whether used individually or in combination, they consistently achieve higher accuracy across all agent types."
}