@inproceedings{fang-etal-2025-cart,
    title = "{CART}: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling",
    author = "Fang, Minghui  and
      Ji, Shengpeng  and
      Zuo, Jialong  and
      Huang, Hai  and
      Xia, Yan  and
      Zhu, Jieming  and
      Cheng, Xize  and
      Yang, Xiaoda  and
      Liu, Wenrui  and
      Wang, Gang  and
      Dong, Zhenhua  and
      Zhao, Zhou",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.735/",
    pages = "15120--15133",
    ISBN = "979-8-89176-251-0",
    abstract = "Cross-modal retrieval aims to search for instances, which are semantically related to the query through the interaction of different modal data. Traditional solutions utilize a single-tower or dual-tower framework to explicitly compute the score between queries and candidates, which is challenged by training cost and inference latency with large-scale data. Inspired by the remarkable performance and efficiency of generative models, we propose a generative cross-modal retrieval framework (CART) based on coarse-to-fine semantic modeling, which assigns identifiers to each candidate and treats the generating identifier as the retrieval target. Specifically, we explore an effective coarse-to-fine scheme, combining K-Means and RQ-VAE to discretize multimodal data into token sequences that support autoregressive generation. Further, considering the lack of explicit interaction between queries and candidates, we propose a feature fusion strategy to align their semantics. Extensive experiments demonstrate the effectiveness of the strategies in the CART, achieving excellent results in both retrieval performance and efficiency."
}