Abstract
The underlying mechanism of neural networks in capturing precise knowledge has been the subject of consistent research efforts. In this work, we propose a theoretical approach based on Neural Tangent Kernels (NTKs) to investigate such mechanisms. Specifically, considering the infinite network width, we hypothesize the learning dynamics of target models may intuitively unravel the features they acquire from training data, deepening our insights into their internal mechanisms. We apply our approach to several fundamental models and reveal how these models leverage statistical features during gradient descent and how they are integrated into final decisions. We also discovered that the choice of activation function can affect feature extraction. For instance, the use of the ReLU activation function could potentially introduce a bias in features, providing a plausible explanation for its replacement with alternative functions in recent pre-trained language models. Additionally, we find that while self-attention and CNN models may exhibit limitations in learning n-grams, multiplication-based models seem to excel in this area. We verify these theoretical findings through experiments and find that they can be applied to analyze language modeling tasks, which can be regarded as a special variant of classification. Our work may offer insights into the roles and capacities of fundamental modules within deep neural networks including large language models.- Anthology ID:
- 2023.emnlp-main.650
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 10505–10530
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.650
- DOI:
- 10.18653/v1/2023.emnlp-main.650
- Cite (ACL):
- Xiaobing Sun, Jiaxi Li, and Wei Lu. 2023. Unraveling Feature Extraction Mechanisms in Neural Networks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10505–10530, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Unraveling Feature Extraction Mechanisms in Neural Networks (Sun et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/fix-dup-bibkey/2023.emnlp-main.650.pdf