@inproceedings{akula-etal-2024-prism,
    title = "{PRISM}: A New Lens for Improved Color Understanding",
    author = "Akula, Arjun Reddy  and
      Pruthi, Garima  and
      Dhillon, Inderjit S  and
      Narayana, Pradyumna  and
      Basu, Sugato  and
      Jampani, Varun",
    editor = "Dernoncourt, Franck  and
      Preo{\c{t}}iuc-Pietro, Daniel  and
      Shimorina, Anastasia",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track",
    month = nov,
    year = "2024",
    address = "Miami, Florida, US",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest_wac_2008/2024.emnlp-industry.121/",
    doi = "10.18653/v1/2024.emnlp-industry.121",
    pages = "1659--1670",
    abstract = "While image-text pre-trained models, such as CLIP, have demonstrated impressive capabilities in learning robust text and image representations, a critical area for substantial improvement remains{---}precise color understanding. In this paper, we address this limitation by introducing PRISM, a simple yet highly effective method that extends CLIP`s capability to grasp the nuances of precise colors. PRISM seamlessly adapts to both recognized HTML colors and out-of-vocabulary RGB inputs through the utilization of our curated dataset of 100 image-text pairs, which can be effortlessly repurposed for fine-tuning with any desired color. Importantly, PRISM achieves these enhancements without compromising CLIP`s performance on established benchmarks. Furthermore, we introduce a novel evaluation framework, ColorLens, featuring both seen and unseen test sets that can be readily repurposed to assess a model`s precision in understanding precise colors. Our comprehensive evaluation and results demonstrate significant improvements over baseline models."
}