Licon: A Diverse, Controllable and Challenging Linguistic Concept Learning Benchmark

Shenglong Yu, Ying Zhang, Wenya Guo, Zhengkun Zhang, Ru Zhou, Xiaojie Yuan

Abstract
Concept Learning requires learning the definition of a general category from given training examples. Most of the existing methods focus on learning concepts from images. However, the visual information cannot present abstract concepts exactly, which struggles the introduction of novel concepts related to known concepts (e.g., ‘Plant’→‘Asteroids’). In this paper, inspired by the fact that humans learn most concepts through linguistic description, we introduce Linguistic Concept Learning benchmark (Licon), where concepts in diverse forms (e.g., plain attributes, images, and text) are defined by linguistic descriptions. The difficulty to learn novel concepts can be controlled by the number of attributes or the hierarchical relationships between concepts. The diverse and controllable concepts are used to support challenging evaluation tasks, including concept classification, attribute prediction, and concept relationship recognition. In addition, we design an entailment-based concept learning method (EnC) to model the relationship among concepts. Extensive experiments demonstrate the effectiveness of EnC. The benchmark will be released to the public soon.
Anthology ID:
2023.findings-emnlp.358
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2023
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5383–5398
Language:
URL:
https://aclanthology.org/2023.findings-emnlp.358
DOI:
10.18653/v1/2023.findings-emnlp.358
Bibkey:
Cite (ACL):
Shenglong Yu, Ying Zhang, Wenya Guo, Zhengkun Zhang, Ru Zhou, and Xiaojie Yuan. 2023. Licon: A Diverse, Controllable and Challenging Linguistic Concept Learning Benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5383–5398, Singapore. Association for Computational Linguistics.
Cite (Informal):
Licon: A Diverse, Controllable and Challenging Linguistic Concept Learning Benchmark (Yu et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-2023-videos/2023.findings-emnlp.358.pdf
PDF Search