Ilaria Torre


2024

pdf bib
Examining Cognitive Biases in ChatGPT 3.5 and ChatGPT 4 through Human Evaluation and Linguistic Comparison
Giada Pantana | Marta Castello | Ilaria Torre
Proceedings of the 16th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)

This paper aims to investigate the presence of cognitive biases, more specifically of Availability heuristics, Representativeness heuristics and Framing, in OpenAI’s ChatGPT 3.5 and ChatGPT 4, as well as the linguistic dependency of their occurrences in the Large Language Models’ (LLMs) outputs. The innovative aspect of this research is conveyed by rephrasing three tasks proposed in Kahneman and Tversky’s works and determining whether the LLMs’ answers to the tasks are correct or incorrect and human-like or non-human-like. The latter classification is made possible by interviewing a total of 56 native speakers of Italian, English and Spanish, thus introducing a new linguistic comparison of results and forming a “human standard’. Our study indicates that GPTs 3.5 and 4 are very frequently subject to the cognitive biases under discussion and their answers are mostly non-human-like. There is minimal but significant discrepancy in the performance of GPT 3.5 and 4, slightly favouring ChatGPT 4 in avoiding biased responses, specifically for Availability heuristics. We also reveal that, while the results for ChatGPT 4 are not significantly language dependent, meaning that the performances in avoiding biases are not affected by the prompting language, their difference with ChatGPT 3.5 is statistically significant.

2019

pdf bib
Prerequisite or Not Prerequisite? That’s the Problem! An NLP-based Approach for Concept Prerequisite Learning
Chiara Alzetta | Alessio Miaschi | Giovanni Adorni | Felice Dell’Orletta | Frosina Koceva | Samuele Passalacqua | Ilaria Torre
Proceedings of the Sixth Italian Conference on Computational Linguistics (CLiC-it 2019)

2018

pdf bib
PRET: Prerequisite-Enriched Terminology. A Case Study on Educational Texts.
Chiara Alzetta | Frosina Koceva | Samuele Passalacqua | Ilaria Torre | Giovanni Adorni
Proceedings of the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018)