@inproceedings{stroligo-etal-2026-evaluation,
    title = "An Evaluation of Classifiers for Mapping Generative {LLM} Responses to Answer Options of Multiple-choice Questionnaires",
    author = "Stroligo, Alisea  and
      Shamray, Anna  and
      Schelb, Julian  and
      Spitz, Andreas",
    editor = "Baez Santamaria, Selene  and
      Somayajula, Sai Ashish  and
      Yamaguchi, Atsuki",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 4: Student Research Workshop)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.eacl-srw.64/",
    pages = "861--880",
    ISBN = "979-8-89176-383-8",
    abstract = "The use of large language models (LLMs) for generating responses to multiple-choice style questionnaires that were originally intended to be answered by humans is often a helpful or even necessary task, for example in persona simulation or during LLM alignment. Although the input and output versatility of generative LLMs is beneficial when adapting such questionnaires to machine use, it can be detrimental when mapping the generated text back to a closed set of possible answer options for evaluation or scoring. In this paper, we investigate the performance of smaller models for the classification of LLM outputs into the available answer options of multiple-choice questionnaires. We consider fine-tuned encoder-transformers as well as a rule-based approach on three datasets with differing answer option complexity. Surprisingly, we find that the best-performing neural approach still underperforms in comparison to our rule-based baseline, indicating that simple pattern-matching of answer options against LLM outputs might still be the most competitive solution for cleaning LLM responses to multiple-choice questionnaires."
}