@inproceedings{daniilidou-etal-2026-social,
    title = "When social robots see our sketches: evaluating human perception of a robot and a {VLM} model performance in a drawing task",
    author = "Daniilidou, Viktoria Paraskevi  and
      Ilinykh, Nikolai  and
      Maraev, Vladislav",
    editor = "Riccardi, Giuseppe  and
      Mousavi, Seyed Mahed  and
      Torres, Maria Ines  and
      Yoshino, Koichiro  and
      Callejas, Zoraida  and
      Chowdhury, Shammur Absar  and
      Chen, Yun-Nung  and
      Bechet, Frederic  and
      Gustafson, Joakim  and
      Damnati, G{\'e}raldine  and
      Papangelis, Alex  and
      D{'}Haro, Luis Fernando  and
      Mendon{\c{c}}a, John  and
      Bernardi, Raffaella  and
      Hakkani-Tur, Dilek  and
      Di Fabbrizio, Giuseppe {''}Pino{''}  and
      Kawahara, Tatsuya  and
      Alam, Firoj  and
      Tur, Gokhan  and
      Johnston, Michael",
    booktitle = "Proceedings of the 16th International Workshop on Spoken Dialogue System Technology",
    month = feb,
    year = "2026",
    address = "Trento, Italy",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/dashboard-stats/2026.iwsds-1.25/",
    pages = "239--252",
    abstract = "We introduce a multimodal framework for interactive drawing in a robot-assisted second language learning scenario. In this scenario, humans are asked to draw objects and spatial relations between them, while a social robot that uses a vision-language model ({VLM}) to analyse whether the drawings are correct.{T}he correctness decision that is passed to the human is coming from a {W}izard-of-{O}z ({W}o{Z}) setup. Therefore, we use it to indirectly evaluate the quality of {VLM} predictions. We show that the task is very challenging for a {VLM} and approaching evaluation of {VLM} performance is important: focusing on the correctness of prediction of certain features (objects, relations) provides a different evaluation picture from when the model is evaluated on prediction of the content of the image as a whole. We also examine how the appearance of the social agent and the type of feedback influence perception of the agent by the participants through a questionnaire. The comparison of verbal feedback, generated by the large language models, against simple pattern-based feedback did not show any significant effects whereas the robot{'}s appearance change indicated significant difference in user ratings concerning naturalness of the agent and its social presence."
}