Abstract
The need to deploy large-scale pre-trained models on edge devices under limited computational resources has led to substantial research to compress these large models. However, less attention has been given to compress the task-specific models. In this work, we investigate the different methods of unstructured pruning on task-specific models for Aspect-based Sentiment Analysis (ABSA) tasks. Specifically, we analyze differences in the learning dynamics of pruned models by using the standard pruning techniques to achieve high-performing sparse networks. We develop a hypothesis to demonstrate the effectiveness of local pruning over global pruning considering a simple CNN model. Later, we utilize the hypothesis to demonstrate the efficacy of the pruned state-of-the-art model compared to the over-parameterized state-of-the-art model under two settings, the first considering the baselines for the same task used for generating the hypothesis, i.e., aspect extraction and the second considering a different task, i.e., sentiment analysis. We also provide discussion related to the generalization of the pruning hypothesis.- Anthology ID:
- 2021.ranlp-srw.17
- Volume:
- Proceedings of the Student Research Workshop Associated with RANLP 2021
- Month:
- September
- Year:
- 2021
- Address:
- Online
- Editors:
- Souhila Djabri, Dinara Gimadi, Tsvetomila Mihaylova, Ivelina Nikolova-Koleva
- Venue:
- RANLP
- SIG:
- Publisher:
- INCOMA Ltd.
- Note:
- Pages:
- 118–125
- Language:
- URL:
- https://aclanthology.org/2021.ranlp-srw.17
- DOI:
- Cite (ACL):
- Abhishek Kumar Mishra and Mohna Chakraborty. 2021. Does local pruning offer task-specific models to learn effectively ?. In Proceedings of the Student Research Workshop Associated with RANLP 2021, pages 118–125, Online. INCOMA Ltd..
- Cite (Informal):
- Does local pruning offer task-specific models to learn effectively ? (Mishra & Chakraborty, RANLP 2021)
- PDF:
- https://preview.aclanthology.org/bionlp-24-ingestion/2021.ranlp-srw.17.pdf
- Code
- abhishekkumarm98/local_vs_global-pruning