@inproceedings{he-etal-2025-flow2code,
    title = "{F}low2{C}ode: Evaluating Large Language Models for Flowchart-based Code Generation Capability",
    author = "He, Mengliang  and
      Zeng, Jiayi  and
      Jiang, Yankai  and
      Zhang, Wei  and
      Liu, Zeming  and
      Shi, Xiaoming  and
      Zhou, Aimin",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.findings-acl.425/",
    pages = "8124--8146",
    ISBN = "979-8-89176-256-5",
    abstract = "While large language models (LLMs) show promise in code generation, existing benchmarks neglect the flowchart-based code generation. To promote further research on flowchart-based code generation, this work presents Flow2Code, a novel benchmark for flowchart-based code generation evaluation. The evaluation dataset spans 15 programming languages and includes 5,622 code segments paired with 16,866 flowcharts of three types: code, UML, and pseudocode. Extensive experiments with 13 multimodal LLMs reveal that current LLMs can not generate code based on flowcharts perfectly. Besides, experiment results show that the supervised fine-tuning technique contributes greatly to the models' performance. The dataset will be publicly available."
}