@inproceedings{wang-etal-2024-telechat,
    title = "{T}ele{C}hat: An Open-source Billingual Large Language Model",
    author = "Wang, Zihan  and
      Liuxz2@chinatelecom.cn, Liuxz2@chinatelecom.cn  and
      Liusx14@chinatelecom.cn, Liusx14@chinatelecom.cn  and
      Yao, Yitong  and
      Huangyy121@chinatelecom.cn, Huangyy121@chinatelecom.cn  and
      Mengxiang, Li  and
      He, Zhongjiang  and
      Liyx25@chinatelecom.cn, Liyx25@chinatelecom.cn  and
      Pulw@chinatelecom.cn, Pulw@chinatelecom.cn  and
      Xuhn@chinatelecom.cn, Xuhn@chinatelecom.cn  and
      Wang, Chao  and
      Song, Shuangyong",
    editor = "Wong, Kam-Fai  and
      Zhang, Min  and
      Xu, Ruifeng  and
      Li, Jing  and
      Wei, Zhongyu  and
      Gui, Lin  and
      Liang, Bin  and
      Zhao, Runcong",
    booktitle = "Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.sighan-1.2",
    pages = "10--20",
    abstract = "In this paper, we present \textbf{TeleChat}, a collection of large language models (LLMs) with parameters of 7 billion and 12 billion. TeleChat is initially pretrained on an extensive corpus containing a diverse collection of texts from both English and Chinese languages, encompassing trillions of tokens. Subsequently, the model undergoes fine-tuning to align with human preferences, following a detailed methodology that we describe. We evaluate the performance of TeleChat on various tasks, including general dialogue generation, language understanding, mathematics, reasoning, code generation, and knowledge-based question answering. Our findings indicate that TeleChat achieves state-of-the-art performance to other open-source models of similar size across a wide range of public benchmarks. To support future research and applications utilizing LLMs, we release the fine-tuned model checkpoints of TeleChat-7B and TeleChat-12B, along with code and a portion of our filtered high-quality pretraining data, to the public community.",
}