Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks
Chengyue Jiang, Yinggong Zhao, Shanbo Chu, Libin Shen, Kewei Tu
Abstract
Neural networks can achieve impressive performance on many natural language processing applications, but they typically need large labeled data for training and are not easily interpretable. On the other hand, symbolic rules such as regular expressions are interpretable, require no training, and often achieve decent accuracy; but rules cannot benefit from labeled data when available and hence underperform neural networks in rich-resource scenarios. In this paper, we propose a type of recurrent neural networks called FA-RNNs that combine the advantages of neural networks and regular expression rules. An FA-RNN can be converted from regular expressions and deployed in zero-shot and cold-start scenarios. It can also utilize labeled data for training to achieve improved prediction accuracy. After training, an FA-RNN often remains interpretable and can be converted back into regular expressions. We apply FA-RNNs to text classification and observe that FA-RNNs significantly outperform previous neural approaches in both zero-shot and low-resource settings and remain very competitive in rich-resource settings.- Anthology ID:
- 2020.emnlp-main.258
- Volume:
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3193–3207
- Language:
- URL:
- https://aclanthology.org/2020.emnlp-main.258
- DOI:
- 10.18653/v1/2020.emnlp-main.258
- Cite (ACL):
- Chengyue Jiang, Yinggong Zhao, Shanbo Chu, Libin Shen, and Kewei Tu. 2020. Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3193–3207, Online. Association for Computational Linguistics.
- Cite (Informal):
- Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks (Jiang et al., EMNLP 2020)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2020.emnlp-main.258.pdf