@inproceedings{zhang-lyu-2025-language,
    title = "Can Language Models Serve as Analogy Annotators?",
    author = "Zhang, Xiaojing  and
      Lyu, Bochen",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.819/",
    pages = "15853--15883",
    ISBN = "979-8-89176-256-5",
    abstract = "Conceptual abstraction and analogy-making are crucial for human learning, reasoning, and adapting to unfamiliar domains. Recently, large language models (LLMs) have made the synthesis of analogical data possible, which, however, still heavily relies on extensive human efforts to be annotated. This paper empirically examines the LLMs' capability to annotate story-level analogical data. Specifically, we propose a novel multi-stage progressive reasoning prompt framework $\texttt{A3E}$ (Automated Analogy Annotation Expert), which is based on the structure mapping theory from cognitive psychology and efficiently annotates candidate story pairs across six fine-grained categories. We use $\texttt{A3E}$ to evaluate how well the state-of-the-art LLMs can serve as analogy annotators. Experimental results demonstrate that our proposed $\texttt{A3E}$ achieves an average performance gain of + 73{\%} across a range of prompting baselines and base LLMs. The code and data is available at https://github.com/zhangxjohn/A3E."
}