@inproceedings{bechard-etal-2026-starflow,
    title = "{S}tar{F}low: Generating Structured Workflow Outputs From Sketch Images",
    author = "Bechard, Patrice  and
      Wang, Chao  and
      Abaskohi, Amirhossein  and
      Rodriguez, Juan A.  and
      Pal, Christopher  and
      Vazquez, David  and
      Gella, Spandana  and
      Rajeswar, Sai  and
      Taslakian, Perouz",
    editor = "Demberg, Vera  and
      Inui, Kentaro  and
      Marquez, Llu{\'i}s",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.76/",
    pages = "1628--1645",
    ISBN = "979-8-89176-380-7",
    abstract = "Workflows are a fundamental component of automation in enterprise platforms, enabling the orchestration of tasks, data processing, and system integrations. Despite being widely used, building workflows can be complex, often requiring manual configuration through low-code platforms or visual programming tools. To simplify this process, we explore the use of generative foundation models, particularly vision-language models (VLMs), to automatically generate structured workflows from visual inputs. Translating hand-drawn sketches or computer-generated diagrams into executable workflows is challenging due to the ambiguity of free-form drawings, variations in diagram styles, and the difficulty of inferring execution logic from visual elements. To address this, we introduce StarFlow, a framework for generating structured workflow outputs from sketches using vision-language models. We curate a diverse dataset of workflow diagrams {--} including synthetic, manually annotated, and real-world samples {--} to enable robust training and evaluation. We finetune and benchmark multiple vision-language models, conducting a series of ablation studies to analyze the strengths and limitations of our approach. Our results show that finetuning significantly enhances structured workflow generation, outperforming large vision-language models on this task."
}