Abstract
Deep neural network based machine learning models are shown to perform poorly on unseen or out-of-domain examples by numerous recent studies. Transfer learning aims to avoid overfitting and to improve generalizability by leveraging the information obtained from multiple tasks. Yet, the benefits of transfer learning depend largely on task selection and finding the right method of sharing. In this thesis, we hypothesize that current deep neural network based transfer learning models do not achieve their fullest potential for various tasks and there are still many task combinations that will benefit from transfer learning that are not considered by the current models. To this end, we started our research by implementing a novel multi-task learner with relaxed annotated data requirements and obtained a performance improvement on two NLP tasks. We will further devise models to tackle tasks from multiple areas of machine learning, such as Bioinformatics and Computer Vision, in addition to NLP.- Anthology ID:
- 2020.acl-srw.6
- Volume:
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop
- Month:
- July
- Year:
- 2020
- Address:
- Online
- Editors:
- Shruti Rijhwani, Jiangming Liu, Yizhong Wang, Rotem Dror
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 33–41
- Language:
- URL:
- https://aclanthology.org/2020.acl-srw.6
- DOI:
- 10.18653/v1/2020.acl-srw.6
- Cite (ACL):
- Arda Akdemir. 2020. Research on Task Discovery for Transfer Learning in Deep Neural Networks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 33–41, Online. Association for Computational Linguistics.
- Cite (Informal):
- Research on Task Discovery for Transfer Learning in Deep Neural Networks (Akdemir, ACL 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2020.acl-srw.6.pdf