Liyuan Mao
2026
Why Can Distillation Work with Limited Resources? A Systematic Study
Xiao Hu | Xingyu Lu | Liyuan Mao | YiFan Zhang | Tianke Zhang | Bin Wen | Fan Yang | Tingting Gao | Guorui Zhou
Findings of the Association for Computational Linguistics: ACL 2026
Xiao Hu | Xingyu Lu | Liyuan Mao | YiFan Zhang | Tianke Zhang | Bin Wen | Fan Yang | Tingting Gao | Guorui Zhou
Findings of the Association for Computational Linguistics: ACL 2026
Recently, large language models have made remarkable progress in reasoning, largely driven by scaling data and model size. In parallel, several studies argue that for smaller models, high-quality distillation can yield strong reasoning performance with minimal resources. However, a framework for understanding machine reasoning that explains why low-resource distillation can boost model performance is still missing. In this paper, we conduct a controlled case study: using less than 920 examples, a simple distillation based on the base model can actually achieve notable reasoning performance improvement, compared with the base model and even the zero-RL models. By analyzing the token frequency in model outputs, we find that the distilled model shows more flexible reasoning. It uses anthropomorphic tokens and logical connectors much more often than the base and zero-RL model. Further analysis reveals that distillation enhances the presence of two advanced cognitive behaviors: Multi-Perspective Thinking or Attempting and Metacognitive Awareness. Frequent occurrences of these two advanced cognitive behaviors give rise to flexible reasoning, which is essential for solving reasoning problems.
2017
Word Embedding and Topic Modeling Enhanced Multiple Features for Content Linking and Argument / Sentiment Labeling in Online Forums
Lei Li | Liyuan Mao | Moye Chen
Proceedings of the MultiLing 2017 Workshop on Summarization and Summary Evaluation Across Source Types and Genres
Lei Li | Liyuan Mao | Moye Chen
Proceedings of the MultiLing 2017 Workshop on Summarization and Summary Evaluation Across Source Types and Genres
Multiple grammatical and semantic features are adopted in content linking and argument/sentiment labeling for online forums in this paper. There are mainly two different methods for content linking. First, we utilize the deep feature obtained from Word Embedding Model in deep learning and compute sentence similarity. Second, we use multiple traditional features to locate candidate linking sentences, and then adopt a voting method to obtain the final result. LDA topic modeling is used to mine latent semantic feature and K-means clustering is implemented for argument labeling, while features from sentiment dictionaries and rule-based sentiment analysis are integrated for sentiment labeling. Experimental results have shown that our methods are valid.