jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models

Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, Samuel R. Bowman


Abstract
We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. jiant enables modular and configuration driven experimentation with state-of-the-art models and a broad set of tasks for probing, transfer learning, and multitask training experiments. jiant implements over 50 NLU tasks, including all GLUE and SuperGLUE benchmark tasks. We demonstrate that jiant reproduces published performance on a variety of tasks and models, e.g., RoBERTa and BERT.
Anthology ID:
2020.acl-demos.15
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
Month:
July
Year:
2020
Address:
Online
Editors:
Asli Celikyilmaz, Tsung-Hsien Wen
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
109–117
Language:
URL:
https://aclanthology.org/2020.acl-demos.15
DOI:
10.18653/v1/2020.acl-demos.15
Bibkey:
Cite (ACL):
Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, and Samuel R. Bowman. 2020. jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 109–117, Online. Association for Computational Linguistics.
Cite (Informal):
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models (Pruksachatkun et al., ACL 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl-24-ws-corrections/2020.acl-demos.15.pdf
Video:
 http://slideslive.com/38928595
Code
 nyu-mll/jiant +  additional community code
Data
BoolQCommonsenseQAGLUEHellaSwagMultiNLISQuADSWAGSuperGLUE