@inproceedings{zheng-etal-2026-masktab,
    title = "{M}ask{T}ab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification",
    author = "Zheng, Bo  and
      Chen, Yudong  and
      Xiong, Zihua  and
      Fang, Shuai  and
      He, Peidong  and
      Yang, Yang  and
      Guo, Sheng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.2053/",
    pages = "41268--41280",
    ISBN = "979-8-89176-395-1",
    abstract = "Tabular data forms the backbone of high-stakes decision systems in finance, healthcare, and beyond. Yet industrial tabular datasets are inherently difficult: high-dimensional, riddled with missing entries, and rarely labeled at scale. While foundation models have revolutionized vision and language, tabular learning still leans on handcrafted features and lacks a general self-supervised framework. We present MaskTab, a unified pre-training framework designed specifically for industrial-scale tabular data. MaskTab encodes missing values via dedicated learnable tokens, enabling the model to distinguish structural absence from random dropout. It jointly optimizes a hybrid supervised pre-training scheme{---}utilizing a twin-path architecture to reconcile masked reconstruction with task-specific supervision{---}and an MoE-augmented loss that adaptively routes features through specialized subnetworks. On industrial-scale benchmarks, it achieves +5.04{\%} AUC and +8.28{\%} KS over prior art under rigorous scaling. Moreover, its representations distill effectively into lightweight models, yielding +2.55{\%} AUC and +4.85{\%} KS under strict latency and interpretability constraints, while improving robustness to distribution shifts. Our work demonstrates that tabular data admits a foundation-model treatment{---}when its structural idiosyncrasies are respected."
}