@inproceedings{xu-zhang-2026-language,
    title = "Language of Thought Shapes Output Diversity in Large Language Models",
    author = "Xu, Shaoyang  and
      Zhang, Wenxuan",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.628/",
    pages = "13802--13816",
    ISBN = "979-8-89176-390-6",
    abstract = "Output diversity is crucial for Large Language Models as it underpins pluralism and creativity.In this work, we reveal that controlling the language used during model thinking{---}the *language of thought*{---}provides a novel and structural source of output diversity.Our preliminary study shows that different thinking languages occupy distinct regions in a model{'}s thinking space.Based on this observation, we study two repeated sampling strategies under multilingual thinking{---}*Single-Language Sampling* and *Mixed-Language Sampling*{---}and conduct diversity evaluation on outputs that are controlled to be in English, regardless of the thinking language used.Across extensive experiments, we demonstrate that switching the thinking language from English to non-English languages consistently increases output diversity, with a clear and consistent positive correlation such that languages farther from English in the thinking space yield larger gains.We further show that aggregating samples across multiple thinking languages yields additional improvements through compositional effects, and that scaling sampling with linguistic heterogeneity expands the model{'}s diversity ceiling.Finally, we show that these findings translate into practical benefits in pluralistic alignment scenarios, leading to broader coverage of cultural knowledge and value orientations in LLM outputs. Our code is publicly available at https://github.com/iNLP-Lab/Multilingual-LoT-Diversity."
}