Training-Free Text Emotion Tagging via LLM-Based Best-Worst Scaling

Lukas Christ, Shahin Amiriparian


Abstract
Large Language Models (LLMs) have been frequently used as automatic annotators for tasks such as Text Emotion Recognition (TER). We consider a scenario in which annotators assign at least one emotion label from a large set of options to a text snippet. For this emotion tagging task, we propose a novel zero-shot algorithm that leverages Best-Worst Scaling (BWS), prompting the LLM to choose the least and most suitable emotions for a given text from several label subsets. The LLM’s choices can be represented by a graph linking labels via worse-than relations. Random walks on this graph yield the final score for each label. We compare our algorithm with naive prompting approaches as well as an established BWS-based method. Extensive experiments demonstrate the suitability of the method. It proves to compare favorably to the benchmarks in terms of both accuracy and calibration with respect to human annotations. Moreover, our algorithm’s automatic annotations are shown to be suitable for finetuning lightweight emotion classification models. The proposed method consumes considerably fewer computational resources than the established BWS approach.
Anthology ID:
2026.findings-eacl.351
Volume:
Findings of the Association for Computational Linguistics: EACL 2026
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Demberg, Kentaro Inui, Lluís Marquez
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6678–6694
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.351/
DOI:
Bibkey:
Cite (ACL):
Lukas Christ and Shahin Amiriparian. 2026. Training-Free Text Emotion Tagging via LLM-Based Best-Worst Scaling. In Findings of the Association for Computational Linguistics: EACL 2026, pages 6678–6694, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
Training-Free Text Emotion Tagging via LLM-Based Best-Worst Scaling (Christ & Amiriparian, Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.351.pdf
Checklist:
 2026.findings-eacl.351.checklist.pdf