Abstract
The huge cost of creating labeled training data is a common problem for supervised learning tasks such as sentiment classification. Recent studies showed that pretraining with unlabeled data via a language model can improve the performance of classification models. In this paper, we take the concept a step further by using a conditional language model, instead of a language model. Specifically, we address a sentiment classification task for a tweet analysis service as a case study and propose a pretraining strategy with unlabeled dialog data (tweet-reply pairs) via an encoder-decoder model. Experimental results show that our strategy can improve the performance of sentiment classifiers and outperform several state-of-the-art strategies including language model pretraining.- Anthology ID:
- P18-2121
- Volume:
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
- Month:
- July
- Year:
- 2018
- Address:
- Melbourne, Australia
- Editors:
- Iryna Gurevych, Yusuke Miyao
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 764–770
- Language:
- URL:
- https://aclanthology.org/P18-2121
- DOI:
- 10.18653/v1/P18-2121
- Cite (ACL):
- Toru Shimizu, Nobuyuki Shimizu, and Hayato Kobayashi. 2018. Pretraining Sentiment Classifiers with Unlabeled Dialog Data. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 764–770, Melbourne, Australia. Association for Computational Linguistics.
- Cite (Informal):
- Pretraining Sentiment Classifiers with Unlabeled Dialog Data (Shimizu et al., ACL 2018)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/P18-2121.pdf