@inproceedings{tam-etal-2024-speak,
    title = "Let Me Speak Freely? A Study On The Impact Of Format Restrictions On Large Language Model Performance.",
    author = "Tam, Zhi Rui  and
      Wu, Cheng-Kuang  and
      Tsai, Yi-Lin  and
      Lin, Chieh-Yen  and
      Lee, Hung-yi  and
      Chen, Yun-Nung",
    editor = "Dernoncourt, Franck  and
      Preo{\c{t}}iuc-Pietro, Daniel  and
      Shimorina, Anastasia",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track",
    month = nov,
    year = "2024",
    address = "Miami, Florida, US",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2024.emnlp-industry.91/",
    doi = "10.18653/v1/2024.emnlp-industry.91",
    pages = "1218--1236",
    abstract = "Structured generation, the process of producing content in standardized formats like JSON and XML, is widely utilized in real-world applications to extract key output information from large language models (LLMs).This study investigates whether such constraints on generation space impact LLMs' abilities, including reasoning and domain knowledge comprehension. Specifically, we evaluate LLMs' performance when restricted to adhere to structured formats versus generating free-form responses across various common tasks. Surprisingly, we observe a significant decline in LLMs' reasoning abilities under format restrictions. Furthermore, we find that stricter format constraints generally lead to greater performance degradation in reasoning tasks."
}