Self-Evaluation of Generative AI Prompts for Linguistic Linked Open Data Modelling in Diachronic Analysis
Florentina Armaselu, Chaya Liebeskind, Giedre Valunaite Oleskeviciene
Abstract
This article addresses the question of evaluating generative AI prompts designed for specific tasks such as linguistic linked open data modelling and refining of word embedding results. The prompts were created to assist the pre-modelling phase in the construction of LLODIA, a linguistic linked open data model for diachronic analysis. We present a self-evaluation framework based on the method known in literature as LLM-Eval. The discussion includes prompts related to the RDF-XML conception of the model, and neighbour list refinement, dictionary alignment and contextualisation for the term revolution in French, Hebrew and Lithuanian, as a proof of concept.- Anthology ID:
- 2024.dlnld-1.8
- Volume:
- Proceedings of the Workshop on Deep Learning and Linked Data (DLnLD) @ LREC-COLING 2024
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Gilles Sérasset, Hugo Gonçalo Oliveira, Giedre Valunaite Oleskeviciene
- Venues:
- DLnLD | WS
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 86–91
- Language:
- URL:
- https://aclanthology.org/2024.dlnld-1.8
- DOI:
- Cite (ACL):
- Florentina Armaselu, Chaya Liebeskind, and Giedre Valunaite Oleskeviciene. 2024. Self-Evaluation of Generative AI Prompts for Linguistic Linked Open Data Modelling in Diachronic Analysis. In Proceedings of the Workshop on Deep Learning and Linked Data (DLnLD) @ LREC-COLING 2024, pages 86–91, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- Self-Evaluation of Generative AI Prompts for Linguistic Linked Open Data Modelling in Diachronic Analysis (Armaselu et al., DLnLD-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.dlnld-1.8.pdf