@inproceedings{wang-etal-2024-telechat,
    title = "{T}ele{C}hat: An Open-source Billingual Large Language Model",
    author = "Wang, Zihan  and
      Liu, XinZhang  and
      Liu, Shixuan  and
      Yao, Yitong  and
      Huang, Yunyao  and
      Li, Mengxiang  and
      He, Zhongjiang  and
      Li, Yongxian  and
      Pu, Luwen  and
      Xu, Huinan  and
      Wang, Chao  and
      Song, Shuangyong",
    editor = "Wong, Kam-Fai  and
      Zhang, Min  and
      Xu, Ruifeng  and
      Li, Jing  and
      Wei, Zhongyu  and
      Gui, Lin  and
      Liang, Bin  and
      Zhao, Runcong",
    booktitle = "Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2024.sighan-1.2/",
    pages = "10--20",
    abstract = "In this paper, we present \textbf{TeleChat}, a collection of large language models (LLMs) with parameters of 7 billion and 12 billion. TeleChat is initially pretrained on an extensive corpus containing a diverse collection of texts from both English and Chinese languages, encompassing trillions of tokens. Subsequently, the model undergoes fine-tuning to align with human preferences, following a detailed methodology that we describe. We evaluate the performance of TeleChat on various tasks, including general dialogue generation, language understanding, mathematics, reasoning, code generation, and knowledge-based question answering. Our findings indicate that TeleChat achieves state-of-the-art performance to other open-source models of similar size across a wide range of public benchmarks. To support future research and applications utilizing LLMs, we release the fine-tuned model checkpoints of TeleChat-7B and TeleChat-12B, along with code and a portion of our filtered high-quality pretraining data, to the public community."
}