@inproceedings{lv-etal-2025-exploiting,
    title = "Exploiting Edited Large Language Models as General Scientific Optimizers",
    author = "Lv, Qitan  and
      Liu, Tianyu  and
      Wang, Hong",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Author-page-Marten-During-lu/2025.naacl-long.270/",
    pages = "5212--5237",
    ISBN = "979-8-89176-189-6",
    abstract = "Large language models (LLMs) have been widely adopted in mathematical optimization in scientific scenarios for their extensive knowledge and advanced reasoning capabilities. Existing methods mainly focus on utilizing LLMs to solve optimization problems in a prompt-based manner, which takes observational feedback as additional textual descriptions. However, due to LLM`s **high sensitivity to the prompts** and **tendency to get lost in lengthy prompts**, these methods struggle to effectively utilize the observational feedback from each optimization step, which severely hinders the applications for real-world scenarios. To address these challenges, we propose a conceptually simple and general bi-level optimization method, namely **G**eneral **S**cientific **O**ptimizers (GSO).Specifically, GSO first utilizes inner-level simulators as experimental platforms to evaluate the current solution and provide observational feedback. Then, LLMs serve as knowledgeable and versatile scientists, generating new solutions by refining potential errors from the feedback as the outer-level optimization.Finally, simulations together with the expert knowledge in LLMs are jointly updated with bi-level interactions via model editing.Extensive experiments show that GSO consistently outperforms existing state-of-the-art methods using *six* different LLM backbone on *seven* different tasks, demonstrating the effectiveness and a wide range of applications."
}