@inproceedings{wu-etal-2025-golden,
    title = "Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models",
    author = "Wu, Xiaojun  and
      Liu, Junxi  and
      Su, Huan-Yi  and
      Lin, Zhouchi  and
      Qi, Yiyan  and
      Xu, Chengjin  and
      Su, Jiajun  and
      Zhong, Jiajie  and
      Wang, Fuwei  and
      Wang, Saizhuo  and
      Hua, Fengrui  and
      Li, Jia  and
      Guo, Jian",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1227/",
    doi = "10.18653/v1/2025.findings-emnlp.1227",
    pages = "22544--22560",
    ISBN = "979-8-89176-335-7",
    abstract = "As large language models (LLMs) increasingly permeate the financial sector, there is a pressing need for a standardized method to comprehensively assess their performance. Existing financial benchmarks often suffer from limited language and task coverage, low-quality datasets, and inadequate adaptability for LLM evaluation. To address these limitations, we introduce Golden Touchstone, a comprehensive bilingual benchmark for financial LLMs, encompassing eight core financial NLP tasks in both Chinese and English. Developed from extensive open-source data collection and industry-specific demands, this benchmark thoroughly assesses models' language understanding and generation capabilities. Through comparative analysis of major models such as GPT-4o, Llama3, FinGPT, and FinMA, we reveal their strengths and limitations in processing complex financial information. Additionally, we open-source Touchstone-GPT, a financial LLM trained through continual pre-training and instruction tuning, which demonstrates strong performance on the bilingual benchmark but still has limitations in specific tasks. This research provides a practical evaluation tool for financial LLMs and guides future development and optimization.The source code for Golden Touchstone and model weight of Touchstone-GPT have been made publicly available at \url{https://github.com/IDEA-FinAI/Golden-Touchstone}."
}