@inproceedings{mao-etal-2025-lightweight,
    title = "Lightweight Contenders: Navigating Semi-Supervised Text Mining through Peer Collaboration and Self Transcendence",
    author = "Mao, Qianren  and
      Jiang, Weifeng  and
      Liu, Junnan  and
      Lin, Chenghua  and
      Li, Qian  and
      Wen, Xianqing  and
      Li, Jianxin  and
      Lu, Jinhu",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.findings-naacl.309/",
    pages = "5571--5585",
    ISBN = "979-8-89176-195-7",
    abstract = "The semi-supervised learning (SSL) strategy in lightweight models requires reducing annotated samples and facilitating cost-effective inference. However, the constraint on model parameters, imposed by the scarcity of training labels, limits the SSL performance. In this paper, we introduce PS-NET, a novel framework tailored for semi-supervised text mining with lightweight models. PS-NET incorporates online distillation to train lightweight student models by imitating the Teacher model. It also integrates an ensemble of student peers that collaboratively instruct each other. Additionally, PS-NET implements a constant adversarial perturbation schema to further self-augmentation by progressive generalizing. Our PS-NET, equipped with a 2-layer distilled BERT, exhibits notable performance enhancements over SOTA lightweight SSL frameworks of FLiText and Disco in SSL text classification with extremely rare labelled data."
}