GPT Perdetry Test: Generating new meanings for new words

Nikolay Malkin, Sameera Lanka, Pranav Goel, Sudha Rao, Nebojsa Jojic


Abstract
Human innovation in language, such as inventing new words, is a challenge for pretrained language models. We assess the ability of one large model, GPT-3, to process new words and decide on their meaning. We create a set of nonce words and prompt GPT-3 to generate their dictionary definitions. We find GPT-3 produces plausible definitions that align with human judgments. Moreover, GPT-3’s definitions are sometimes preferred to those invented by humans, signaling its intriguing ability not just to adapt, but to add to the evolving vocabulary of the English language.
Anthology ID:
2021.naacl-main.439
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5542–5553
Language:
URL:
https://aclanthology.org/2021.naacl-main.439
DOI:
10.18653/v1/2021.naacl-main.439
Bibkey:
Cite (ACL):
Nikolay Malkin, Sameera Lanka, Pranav Goel, Sudha Rao, and Nebojsa Jojic. 2021. GPT Perdetry Test: Generating new meanings for new words. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5542–5553, Online. Association for Computational Linguistics.
Cite (Informal):
GPT Perdetry Test: Generating new meanings for new words (Malkin et al., NAACL 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.naacl-main.439.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2021.naacl-main.439.mp4