Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text

Lukas Ruff, Yury Zemlyanskiy, Robert Vandermeulen, Thomas Schnake, Marius Kloft


Abstract
There exist few text-specific methods for unsupervised anomaly detection, and for those that do exist, none utilize pre-trained models for distributed vector representations of words. In this paper we introduce a new anomaly detection method—Context Vector Data Description (CVDD)—which builds upon word embedding models to learn multiple sentence representations that capture multiple semantic contexts via the self-attention mechanism. Modeling multiple contexts enables us to perform contextual anomaly detection of sentences and phrases with respect to the multiple themes and concepts present in an unlabeled text corpus. These contexts in combination with the self-attention weights make our method highly interpretable. We demonstrate the effectiveness of CVDD quantitatively as well as qualitatively on the well-known Reuters, 20 Newsgroups, and IMDB Movie Reviews datasets.
Anthology ID:
P19-1398
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4061–4071
Language:
URL:
https://aclanthology.org/P19-1398
DOI:
10.18653/v1/P19-1398
Bibkey:
Cite (ACL):
Lukas Ruff, Yury Zemlyanskiy, Robert Vandermeulen, Thomas Schnake, and Marius Kloft. 2019. Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4061–4071, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text (Ruff et al., ACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-2/P19-1398.pdf
Code
 lukasruff/CVDD-PyTorch