Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models
Georg Ahnert, Anna-Carolina Haensch, Barbara Plank, Markus Strohmaier
Abstract
Many _in-silico_ simulations of human survey responses with large language models (LLMs) focus on generating closed-ended survey responses, whereas LLMs are typically trained to generate open-ended text instead. Previous research has used a diverse range of methods for generating closed-ended survey responses with LLMs and a standard practice remains to be identified. In this paper, we systematically investigate the impact that various **Survey Response Generation Methods** have on predicted survey responses. We present the results of 32 mio. simulated survey responses across 8 Survey Response Generation Methods, 4 political attitude surveys, and 10 open-weight language models. We find significant differences between the Survey Response Generation Methods in both individual-level and subpopulation-level alignment. Our results show that Restricted Generation Methods perform best overall, and that reasoning output does not consistently improve alignment. Our work underlines the significant impact that Survey Response Generation Methods have on simulated survey responses, and we develop practical recommendations on the application of Survey Response Generation Methods.- Anthology ID:
- 2026.acl-long.1927
- Volume:
- Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 41554–41577
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl/2026.acl-long.1927/
- DOI:
- Cite (ACL):
- Georg Ahnert, Anna-Carolina Haensch, Barbara Plank, and Markus Strohmaier. 2026. Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 41554–41577, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models (Ahnert et al., ACL 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl/2026.acl-long.1927.pdf