@inproceedings{singh-etal-2026-asking,
    title = "Asking language models how to represent data for fine-tuning",
    author = "Singh, Usneek  and
      Singha, Ananya  and
      Awasthi, Abhijeet  and
      Gulwani, Sumit  and
      Kanade, Aditya  and
      Le, Vu  and
      Singh, Mukul  and
      Verbruggen, Gust",
    editor = "Gupta, Vivek  and
      Ding, Kaize  and
      Kokel, Harsha  and
      Zhao, Yue  and
      Agarwal, Amit  and
      Wang, Yu  and
      Glass, Michael  and
      Zhang, Yu  and
      Srinivas, Kavitha  and
      Chen, Xiusi  and
      Hassanzadeh, Oktie  and
      Zhu, Qi  and
      Chang, Shuaichen  and
      Luo, Yuan",
    booktitle = "Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the {LLM} Era ({SURG}e{LLM} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.surgellm-1.13/",
    pages = "209--218",
    ISBN = "979-8-89176-406-4",
    abstract = "Language models are often used for tasks involving structured data like tables and graphs, but there is no principled approach for choosing the best format to represent such data for fine-tuning. We address this in three steps. First, we show that format choice remains important even after fine-tuning; models learn more efficiently with specific formats rather than adapting to any format. Second, we show that a pre-trained model can suggest its own candidate formats by auto-completing partial prompts, reducing reliance on developer intuition. Third, and most importantly, we demonstrate that base model performance across formats reliably predicts post-fine-tuning performance: the format that performs best before fine-tuning remains among the top candidates after fine-tuning in 16 out of 18 settings across three data structure types, three models, and six tasks. This finding allows format selection to be done via inference alone, avoiding costly trial-and-error fine-tuning runs."
}