Text-Guided Alternative Image Clustering
Andreas Stephan, Lukas Miklautz, Collin Leiber, Pedro Henrique Luz De Araujo, Dominik Répás, Claudia Plant, Benjamin Roth
Abstract
Traditional image clustering techniques only find a single grouping within visual data. In particular, they do not provide a possibility to explicitly define multiple types of clustering. This work explores the potential of large vision-language models to facilitate alternative image clustering. We propose Text-Guided Alternative Image Consensus Clustering (TGAICC), a novel approach that leverages user-specified interests via prompts to guide the discovery of diverse clusterings. To achieve this, it generates a clustering for each prompt, groups them using hierarchical clustering, and then aggregates them using consensus clustering. TGAICC outperforms image- and text-based baselines on four alternative image clustering benchmark datasets. Furthermore, using count-based word statistics, we are able to obtain text-based explanations of the alternative clusterings. In conclusion, our research illustrates how contemporary large vision-language models can transform explanatory data analysis, enabling the generation of insightful, customizable, and diverse image clusterings.- Anthology ID:
- 2024.repl4nlp-1.13
- Volume:
- Proceedings of the 9th Workshop on Representation Learning for NLP (RepL4NLP-2024)
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Chen Zhao, Marius Mosbach, Pepa Atanasova, Seraphina Goldfarb-Tarrent, Peter Hase, Arian Hosseini, Maha Elbayad, Sandro Pezzelle, Maximilian Mozes
- Venues:
- RepL4NLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 177–190
- Language:
- URL:
- https://aclanthology.org/2024.repl4nlp-1.13
- DOI:
- Cite (ACL):
- Andreas Stephan, Lukas Miklautz, Collin Leiber, Pedro Henrique Luz De Araujo, Dominik Répás, Claudia Plant, and Benjamin Roth. 2024. Text-Guided Alternative Image Clustering. In Proceedings of the 9th Workshop on Representation Learning for NLP (RepL4NLP-2024), pages 177–190, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Text-Guided Alternative Image Clustering (Stephan et al., RepL4NLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2024.repl4nlp-1.13.pdf