Evaluating Large Language Models for Linguistic Linked Data Generation
Maria Pia di Buono, Blerina Spahiu, Verginica Barbu Mititelu
Abstract
Large language models (LLMs) have revolutionized human-machine interaction with their ability to converse and perform various language tasks. This study investigates the potential of LLMs for knowledge formalization using well-defined vocabularies, specifically focusing on OntoLex-Lemon. As a preliminary exploration, we test four languages (English, Italian, Albanian, Romanian) and analyze the formalization quality of nine words with varying characteristics applying a multidimensional evaluation approach. While manual validation provided initial insights, it highlights the need for developing scalable evaluation methods for future large-scale experiments. This research aims to initiate a discussion on the potential and challenges of utilizing LLMs for knowledge formalization within the Semantic Web framework.- Anthology ID:
- 2024.dlnld-1.6
- Volume:
- Proceedings of the Workshop on Deep Learning and Linked Data (DLnLD) @ LREC-COLING 2024
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Gilles Sérasset, Hugo Gonçalo Oliveira, Giedre Valunaite Oleskeviciene
- Venues:
- DLnLD | WS
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 66–75
- Language:
- URL:
- https://aclanthology.org/2024.dlnld-1.6
- DOI:
- Cite (ACL):
- Maria Pia di Buono, Blerina Spahiu, and Verginica Barbu Mititelu. 2024. Evaluating Large Language Models for Linguistic Linked Data Generation. In Proceedings of the Workshop on Deep Learning and Linked Data (DLnLD) @ LREC-COLING 2024, pages 66–75, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- Evaluating Large Language Models for Linguistic Linked Data Generation (di Buono et al., DLnLD-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.dlnld-1.6.pdf