@inproceedings{krause-etal-2023-confidently,
    title = "Confidently Wrong: Exploring the Calibration and Expression of (Un)Certainty of Large Language Models in a Multilingual Setting",
    author = "Krause, Lea  and
      Tufa, Wondimagegnhue  and
      Baez Santamaria, Selene  and
      Daza, Angel  and
      Khurana, Urja  and
      Vossen, Piek",
    editor = "Gatt, Albert  and
      Gardent, Claire  and
      Cripwell, Liam  and
      Belz, Anya  and
      Borg, Claudia  and
      Erdem, Aykut  and
      Erdem, Erkut",
    booktitle = "Proceedings of the Workshop on Multimodal, Multilingual Natural Language Generation and Multilingual WebNLG Challenge (MM-NLG 2023)",
    month = sep,
    year = "2023",
    address = "Prague, Czech Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/remove-affiliations/2023.mmnlg-1.1/",
    pages = "1--9",
    abstract = "While the fluency and coherence of Large Language Models (LLMs) in text generation have seen significant improvements, their competency in generating appropriate expressions of uncertainty remains limited.Using a multilingual closed-book QA task and GPT-3.5, we explore how well LLMs are calibrated and express certainty across a diverse set of languages, including low-resource settings. Our results reveal strong performance in high-resource languages but a marked decline in performance in lower-resource languages. Across all, we observe an exaggerated expression of confidence in the model, which does not align with the correctness or likelihood of its responses. Our findings highlight the need for further research into accurate calibration of LLMs especially in a multilingual setting."
}