@inproceedings{luo-etal-2026-llmsurgeon,
    title = "{LLMS}urgeon: Diagnosing Data Mixture of Large Language Models",
    author = "Luo, Yaxin  and
      Cui, Jiacheng  and
      Zhao, Xiaohan  and
      Shang, Xinyi  and
      Liu, Jiacheng  and
      Bi, Xinyue  and
      Li, Zhaoyi  and
      Shen, Zhiqiang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1964/",
    pages = "42444--42459",
    ISBN = "979-8-89176-390-6",
    abstract = "The pretraining data mixture of Large Language Models (LLMs) constitutes their ``digital DNA'', shaping model behaviors, capabilities, and failure modes. Yet this composition is rarely disclosed, making post-hoc auditing of data combination or provenance difficult. In this work, we formalize $\textbf{Data Mixture Surgery (DMS)}$: given only generated text from a target LLM, estimate the domain-level distribution of its pretraining corpus under a predefined taxonomy. We propose $\textbf{LLMSurgeon}$, a strong framework that casts DMS as an inverse problem under the label-shift assumption. Rather than directly aggregating classifier outputs, $LLMSurgeon$ estimates a calibrated $\textit{soft}$ confusion matrix and solves a constrained inverse problem to correct systematic domain confusion and recover the latent mixture prior. To evaluate, we introduce $\textbf{LLMScan}$, a recipe-verifiable evaluation suite built from open-source LLMs with transparent pretraining mixtures. Across LLMScan, $LLMSurgeon$ recovers domain mixtures with high fidelity under fixed protocols. Our work presents a practical, post-hoc approach for auditing the $\textit{digital DNA}$ of foundation models without access to their training data. Code is available at: https://github.com/Yaxin9Luo/LLMSurgeon."
}