@inproceedings{xiao-etal-2025-longweave,
    title = "{L}ong{W}eave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiability",
    author = "Xiao, Zikai  and
      Huang, Fei  and
      Tu, Jianhong  and
      Wei, Jianhui  and
      Ma, Wen  and
      Zhou, Yuxuan  and
      Wu, Jian  and
      Yu, Bowen  and
      Liu, Zuozhu  and
      Lin, Junyang",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/name-variant-enfa-fane/2025.findings-emnlp.549/",
    doi = "10.18653/v1/2025.findings-emnlp.549",
    pages = "10386--10417",
    ISBN = "979-8-89176-335-7",
    abstract = "Generating long, informative, and factual outputs remains a major challenge for Large Language Models (LLMs). Existing benchmarks for long-form generation typically assess real-world queries with hard-to-verify metrics or use synthetic setups that ease evaluation but overlook real-world intricacies. In this paper, we introduce \textbf{LongWeave}, which balance real-world and verifiable assessment with Target-Anchored Evaluation (TAE). TAE constructs tasks by first defining verifiable targets within real-world scenarios, then systematically generating corresponding queries, textual materials, and anchors based on these targets. This ensures that tasks are both realistic and objectively assessable, enabling rigorous assessment of model capabilities in meeting complex real-world constraints. LongWeave supports customizable input/output lengths (up to 64K/8K tokens) across seven distinct tasks. Evaluation on 23 LLMs show that even state-of-the-art models encounter significant challenges in long-form generation as real-world complexity and output length increase. Dataset will be publicly available."
}