@inproceedings{chunyu-etal-2025-ji,
    title = "基于提示探针的大模型知识掌握能力评测",
    author = "Chunyu, Wang  and
      Chen, Bo  and
      Xu, Yang  and
      Zhao, Xiaobing",
    editor = "Sun, Maosong  and
      Duan, Peiyong  and
      Liu, Zhiyuan  and
      Xu, Ruifeng  and
      Sun, Weiwei",
    booktitle = "Proceedings of the 24th {C}hina National Conference on Computational Linguistics ({CCL} 2025)",
    month = aug,
    year = "2025",
    address = "Jinan, China",
    publisher = "Chinese Information Processing Society of China",
    url = "https://preview.aclanthology.org/ingest-ccl/2025.ccl-1.24/",
    pages = "311--322",
    abstract = "``大语言模型在知识密集型任务中的表现高度依赖其内化知识的覆盖面和掌握程度。然而,当前缺乏系统化、细粒度的评测方法以刻画模型对不同类别知识的掌握能力。为此,本文提出一种基于提示探针的方法,系统评估大语言模型在常识性知识、事实性知识和专业领域知识方面的掌握情况。首先构建了一个高质量的知识探针评测数据集KPE-Pro(Knowledge Probing Evaluation for Proficiency)。然后设计提示模板对多个主流大语言模型进行系统评测。评测结果表明,大语言模型在常识性知识方面表现较好,ERNIE X1模型取得整体最好成绩;在事实性知识上,大语言模型的表现较弱,轻量模型的知识掌握能力明显不足。评测数据公开于:https://github.com/cyuu313/KPE-Pro。''"
}