PSK@EEUCA 2026: Fine-tuning Large Language Models with Synthetic Data Augmentation for Multi-class Toxicity Detection in Gaming Chat

Srikar Kashyap Pulipaka


Abstract
This paper describes our system for the EEUCA 2026 Shared Task on Understanding Toxic Behavior in Gaming Communities. The task involves classifying World of Tanks chat messages into six toxicity categories: Non-toxic, Insults/Flaming, Other Offensive, Hate/Harassment, Threats, and Extremism. We explore multiple approaches including encoder-based models, instruction-tuned LLMs with LoRA fine-tuning, hierarchical classification, one-vs-rest strategies, and various ensemble methods. Our best system combines Llama 3.1 8B with carefully calibrated 5% synthetic data augmentation, achieving an F1-macro score of 0.6234 on the test set, placing 4th out of 35 participating teams. We provide extensive analysis of the dataset’s annotation patterns and their impact on model generalization, revealing a critical “validation trap” phenomenon where high validation performance correlates with poor test transfer.
Anthology ID:
2026.eeuca-1.21
Volume:
Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ali Hürriyetoğlu, Surendrabikram Thapa, Hristo Tanev
Venues:
EEUCA | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
192–197
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.eeuca-1.21/
DOI:
Bibkey:
Cite (ACL):
Srikar Kashyap Pulipaka. 2026. PSK@EEUCA 2026: Fine-tuning Large Language Models with Synthetic Data Augmentation for Multi-class Toxicity Detection in Gaming Chat. In Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026), pages 192–197, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
PSK@EEUCA 2026: Fine-tuning Large Language Models with Synthetic Data Augmentation for Multi-class Toxicity Detection in Gaming Chat (Pulipaka, EEUCA 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.eeuca-1.21.pdf