@inproceedings{zhao-etal-2025-sphere,
    title = "{SPHERE}: An Evaluation Card for Human-{AI} Systems",
    author = "Zhao, Dora  and
      Ma, Qianou  and
      Zhao, Xinran  and
      Si, Chenglei  and
      Yang, Chenyang  and
      Louie, Ryan  and
      Reiter, Ehud  and
      Yang, Diyi  and
      Wu, Tongshuang",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.70/",
    pages = "1340--1365",
    ISBN = "979-8-89176-256-5",
    abstract = "In the era of Large Language Models (LLMs), establishing effective evaluation methods and standards for diverse human-AI interaction systems is increasingly challenging. To encourage more transparent documentation and facilitate discussion on human-AI system evaluation design options, we present an evaluation card SPHERE, which encompasses five key dimensions: 1) What is being evaluated?; 2) How is the evaluation conducted?; 3) Who is participating in the evaluation?; 4) When is evaluation conducted?; 5) How is evaluation validated? We conduct a review of 39 human-AI systems using SPHERE, outlining current evaluation practices and areas for improvement. We provide three recommendations for improving the validity and rigor of evaluation practices."
}