Ad Lingua: Text Classification Improves Symbolism Prediction in Image Advertisements
Andrey Savchenko, Anton Alekseev, Sejeong Kwon, Elena Tutubalina, Evgeny Myasnikov, Sergey Nikolenko
Abstract
Understanding image advertisements is a challenging task, often requiring non-literal interpretation. We argue that standard image-based predictions are insufficient for symbolism prediction. Following the intuition that texts and images are complementary in advertising, we introduce a multimodal ensemble of a state of the art image-based classifier, a classifier based on an object detection architecture, and a fine-tuned language model applied to texts extracted from ads by OCR. The resulting system establishes a new state of the art in symbolism prediction.- Anthology ID:
- 2020.coling-main.171
- Volume:
- Proceedings of the 28th International Conference on Computational Linguistics
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona, Spain (Online)
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 1886–1892
- Language:
- URL:
- https://aclanthology.org/2020.coling-main.171
- DOI:
- 10.18653/v1/2020.coling-main.171
- Cite (ACL):
- Andrey Savchenko, Anton Alekseev, Sejeong Kwon, Elena Tutubalina, Evgeny Myasnikov, and Sergey Nikolenko. 2020. Ad Lingua: Text Classification Improves Symbolism Prediction in Image Advertisements. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1886–1892, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Cite (Informal):
- Ad Lingua: Text Classification Improves Symbolism Prediction in Image Advertisements (Savchenko et al., COLING 2020)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2020.coling-main.171.pdf
- Data
- Open Images V4