Yudong Chen
2026
MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification
Bo Zheng | Yudong Chen | Zihua Xiong | Shuai Fang | Peidong He | Yang Yang | Sheng Guo
Findings of the Association for Computational Linguistics: ACL 2026
Bo Zheng | Yudong Chen | Zihua Xiong | Shuai Fang | Peidong He | Yang Yang | Sheng Guo
Findings of the Association for Computational Linguistics: ACL 2026
Tabular data forms the backbone of high-stakes decision systems in finance, healthcare, and beyond. Yet industrial tabular datasets are inherently difficult: high-dimensional, riddled with missing entries, and rarely labeled at scale. While foundation models have revolutionized vision and language, tabular learning still leans on handcrafted features and lacks a general self-supervised framework. We present MaskTab, a unified pre-training framework designed specifically for industrial-scale tabular data. MaskTab encodes missing values via dedicated learnable tokens, enabling the model to distinguish structural absence from random dropout. It jointly optimizes a hybrid supervised pre-training scheme—utilizing a twin-path architecture to reconcile masked reconstruction with task-specific supervision—and an MoE-augmented loss that adaptively routes features through specialized subnetworks. On industrial-scale benchmarks, it achieves +5.04% AUC and +8.28% KS over prior art under rigorous scaling. Moreover, its representations distill effectively into lightweight models, yielding +2.55% AUC and +4.85% KS under strict latency and interpretability constraints, while improving robustness to distribution shifts. Our work demonstrates that tabular data admits a foundation-model treatment—when its structural idiosyncrasies are respected.
2023
Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model
Xinyu Zhou | Delong Chen | Yudong Chen
Proceedings of the 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023)
Xinyu Zhou | Delong Chen | Yudong Chen
Proceedings of the 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023)