@inproceedings{liu-etal-2023-molca,
    title = "{M}ol{CA}: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter",
    author = "Liu, Zhiyuan  and
      Li, Sihang  and
      Luo, Yanchen  and
      Fei, Hao  and
      Cao, Yixin  and
      Kawaguchi, Kenji  and
      Wang, Xiang  and
      Chua, Tat-Seng",
    editor = "Bouamor, Houda  and
      Pino, Juan  and
      Bali, Kalika",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest_wac_2008/2023.emnlp-main.966/",
    doi = "10.18653/v1/2023.emnlp-main.966",
    pages = "15623--15638",
    abstract = "Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception {---} a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter. MolCA enables an LM (i.e., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector. Specifically, the cross-modal projector is implemented as a Q-Former to connect a graph encoder`s representation space and an LM`s text space. Further, MolCA employs a uni-modal adapter (i.e., LoRA) for the LM`s efficient adaptation to downstream tasks. Unlike previous studies that couple an LM with a graph encoder via cross-modal contrastive learning, MolCA retains the LM`s ability of open-ended text generation and augments it with 2D graph information. To showcase its effectiveness, we extensively benchmark MolCA on tasks of molecule captioning, IUPAC name prediction, and molecule-text retrieval, on which MolCA significantly outperforms the baselines."
}