@inproceedings{kruschwitz-schmidhuber-2024-llm,
    title = "{LLM}-Based Synthetic Datasets: Applications and Limitations in Toxicity Detection",
    author = "Schmidhuber, Maximilian  and
      Kruschwitz, Udo",
    editor = "Kumar, Ritesh  and
      Ojha, Atul Kr.  and
      Malmasi, Shervin  and
      Chakravarthi, Bharathi Raja  and
      Lahiri, Bornini  and
      Singh, Siddharth  and
      Ratan, Shyam",
    booktitle = "Proceedings of the Fourth Workshop on Threat, Aggression {\&} Cyberbullying @ LREC-COLING-2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://preview.aclanthology.org/transition-to-people-yaml/2024.trac-1.6/",
    pages = "37--51",
    abstract = "Large Language Model (LLM)-based Synthetic Data is becoming an increasingly important field of research. One of its promising applications is in training classifiers to detect online toxicity, which is of increasing concern in today{'}s digital landscape. In this work, we assess the feasibility of generative models to create synthetic data for toxic language detection. Our experiments are conducted on six different toxicity datasets, four of whom are hateful and two are toxic in the broader sense. We then employ a classifier trained on the original data for filtering. To explore the potential of this data, we conduct experiments using combinations of original and synthetic data, synthetic oversampling of the minority class, and a comparison of original vs. synthetic-only training. Results indicate that while our generative models offer benefits in certain scenarios, the approach does not improve hateful dataset classification. However, it does boost patronizing and condescending language detection. We find that synthetic data generated by LLMs is a promising avenue of research, but further research is needed to improve the quality of the generated data and develop better filtering methods. Code is available on GitHub; the generated dataset is available on Zenodo."
}