Abstract
This paper studies the application of pretrained BERT in the acquisition of synonyms, antonyms, hypernyms and hyponyms in Portuguese. Masked patterns indicating those relations were compiled with the help of a service for validating semantic relations, and then used for prompting three pretrained BERT models, one multilingual and two for Portuguese (base and large). Predictions for the masks were evaluated in two different test sets. Results achieved by the monolingual models are interesting enough for considering these models as a source for enriching wordnets, especially when predicting hypernyms of nouns. Previously reported performances on prediction were improved with new patterns and with the large model. When it comes to selecting the related word from a set of four options, performance is even better, but not enough for outperforming the selection of the most similar word, as computed with static word embeddings.- Anthology ID:
- 2023.gwc-1.5
- Volume:
- Proceedings of the 12th Global Wordnet Conference
- Month:
- January
- Year:
- 2023
- Address:
- University of the Basque Country, Donostia - San Sebastian, Basque Country
- Editors:
- German Rigau, Francis Bond, Alexandre Rademaker
- Venue:
- GWC
- SIG:
- Publisher:
- Global Wordnet Association
- Note:
- Pages:
- 41–49
- Language:
- URL:
- https://aclanthology.org/2023.gwc-1.5
- DOI:
- Cite (ACL):
- Hugo Gonçalo Oliveira. 2023. On the Acquisition of WordNet Relations in Portuguese from Pretrained Masked Language Models. In Proceedings of the 12th Global Wordnet Conference, pages 41–49, University of the Basque Country, Donostia - San Sebastian, Basque Country. Global Wordnet Association.
- Cite (Informal):
- On the Acquisition of WordNet Relations in Portuguese from Pretrained Masked Language Models (Oliveira, GWC 2023)
- PDF:
- https://preview.aclanthology.org/corrections-2024-07/2023.gwc-1.5.pdf