MTRec: Multi-Task Learning over BERT for News Recommendation
Qiwei Bi, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Hanfang Yang
Abstract
Existing news recommendation methods usually learn news representations solely based on news titles. To sufficiently utilize other fields of news information such as category and entities, some methods treat each field as an additional feature and combine different feature vectors with attentive pooling. With the adoption of large pre-trained models like BERT in news recommendation, the above way to incorporate multi-field information may encounter challenges: the shallow feature encoding to compress the category and entity information is not compatible with the deep BERT encoding. In this paper, we propose a multi-task method to incorporate the multi-field information into BERT, which improves its news encoding capability. Besides, we modify the gradients of auxiliary tasks based on their gradient conflicts with the main task, which further boosts the model performance. Extensive experiments on the MIND news recommendation benchmark show the effectiveness of our approach.- Anthology ID:
- 2022.findings-acl.209
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2022
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Smaranda Muresan, Preslav Nakov, Aline Villavicencio
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2663–2669
- Language:
- URL:
- https://aclanthology.org/2022.findings-acl.209
- DOI:
- 10.18653/v1/2022.findings-acl.209
- Cite (ACL):
- Qiwei Bi, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, and Hanfang Yang. 2022. MTRec: Multi-Task Learning over BERT for News Recommendation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2663–2669, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- MTRec: Multi-Task Learning over BERT for News Recommendation (Bi et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2022.findings-acl.209.pdf
- Data
- MIND