@inproceedings{kousa-2023-introducing,
    title = "Introducing {C}hat{GPT} to a researcher{'}s toolkit: An empirical comparison between rule-based and large language model approach in the context of qualitative content analysis of political texts in {F}innish",
    author = "Kousa, Ilona",
    editor = {H{\"a}m{\"a}l{\"a}inen, Mika  and
      {\"O}hman, Emily  and
      Pirinen, Flammie  and
      Alnajjar, Khalid  and
      Miyagawa, So  and
      Bizzoni, Yuri  and
      Partanen, Niko  and
      Rueter, Jack},
    booktitle = "Proceedings of the Joint 3rd International Conference on Natural Language Processing for Digital Humanities and 8th International Workshop on Computational Linguistics for Uralic Languages",
    month = dec,
    year = "2023",
    address = "Tokyo, Japan",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2023.nlp4dh-1.12/",
    pages = "102--113",
    abstract = "Large Language Models, such as ChatGPT, offer numerous possibilities and prospects for academic research. However, there has been a gap in empirical research regarding their utilisation as keyword extraction and classification tools in qualitative research; perspectives from the social sciences and humanities have been notably limited. Moreover, Finnish-language data have not been used in previous studies. In this article, I aim to address these gaps by providing insights into the utilisation of ChatGPT and drawing comparisons with a rule-based Natural Language Processing method called Etuma. I will focus on assessing the effectiveness of classification and the methods' adherence to scientific principles. The findings of the study indicate that the classic recall and precision trade-off applies to the methods: ChatGPT{'}s precision is high, but its recall is comparatively low, while the results are the opposite for Etuma. I also discuss the implications of the results and outline ideas for leveraging the strengths of both methods in future studies."
}