Marc Döbler


2024

pdf
Can I trust You? LLMs as conversational agents
Marc Döbler | Raghavendran Mahendravarman | Anna Moskvina | Nasrin Saef
Proceedings of the 1st Workshop on Personalization of Generative AI Systems (PERSONALIZE 2024)

With the rising popularity of LLMs in the public sphere, they become more and more attractive as a tool for doing one’s own research without having to rely on search engines or specialized knowledge of a scientific field. But using LLMs as a source for factual information can lead one to fall prey to misinformation or hallucinations dreamed up by the model. In this paper we examine the gpt-4 LLM by simulating a large number of potential research queries and evaluate how many of the generated references are factually correct as well as existent.