@inproceedings{li-etal-2023-cleva,
    title = "{CLEVA}: {C}hinese Language Models {EVA}luation Platform",
    author = "Li, Yanyang  and
      Zhao, Jianqiao  and
      Zheng, Duo  and
      Hu, Zi-Yuan  and
      Chen, Zhi  and
      Su, Xiaohui  and
      Huang, Yongfeng  and
      Huang, Shijia  and
      Lin, Dahua  and
      Lyu, Michael  and
      Wang, Liwei",
    editor = "Feng, Yansong  and
      Lefever, Els",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest_wac_2008/2023.emnlp-demo.17/",
    doi = "10.18653/v1/2023.emnlp-demo.17",
    pages = "186--217",
    abstract = "With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model`s capabilities has become an increasingly significant issue. The absence of a comprehensive Chinese benchmark that thoroughly assesses a model`s performance, the unstandardized and incomparable prompting procedure, and the prevalent risk of contamination pose major challenges in the current evaluation of Chinese LLMs. We present CLEVA, a user-friendly platform crafted to holistically evaluate Chinese LLMs. Our platform employs a standardized workflow to assess LLMs' performance across various dimensions, regularly updating a competitive leaderboard. To alleviate contamination, CLEVA curates a significant proportion of new data and develops a sampling strategy that guarantees a unique subset for each leaderboard round. Empowered by an easy-to-use interface that requires just a few mouse clicks and a model API, users can conduct a thorough evaluation with minimal coding. Large-scale experiments featuring 23 Chinese LLMs have validated CLEVA`s efficacy."
}