Jingsheng Zheng
2026
Can We Predict Before Executing Machine Learning Agents?
Jingsheng Zheng | Jintian Zhang | Yujie Luo | Yuren Mao | Yunjun Gao | Lun Du | Huajun Chen | Ningyu Zhang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Jingsheng Zheng | Jintian Zhang | Yujie Luo | Yuren Mao | Yunjun Gao | Lun Du | Huajun Chen | Ningyu Zhang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Autonomous machine learning agents have revolutionized scientific discovery, yet they remain constrained by a Generate-Execute-Feedback paradigm. Previous approaches suffers from a severe Execution Bottleneck, as hypothesis evaluation relies strictly on expensive physical execution. To bypass these physical constraints, we internalize execution priors to substitute costly runtime checks with instantaneous predictive reasoning, drawing inspiration from World Models. In this work, we formalize the task of Data-centric Solution Preference and construct a comprehensive corpus of 18,438 pairwise comparisons. We demonstrate that LLMs exhibit significant predictive capabilities when primed with a Verified Data Analysis Report, achieving 61.5% accuracy and robust confidence calibration. Finally, we instantiate this framework in ForeAgent, an agent that employs a Predict-then-Verify loop, achieving a 6x acceleration in convergence while surpassing execution-based baselines by +6%.
2024
VideoCoT: A Video Chain-of-Thought Dataset with Active Annotation Tool
Yan Wang | Yawen Zeng | Jingsheng Zheng | Xiaofen Xing | Jin Xu | Xiangmin Xu
Proceedings of the 3rd Workshop on Advances in Language and Vision Research (ALVR)
Yan Wang | Yawen Zeng | Jingsheng Zheng | Xiaofen Xing | Jin Xu | Xiangmin Xu
Proceedings of the 3rd Workshop on Advances in Language and Vision Research (ALVR)
Multimodal large language models (MLLMs) are flourishing, but mainly focus on images with less attention than videos, especially in sub-fields such as prompt engineering, video chain-of-though (CoT), and instruction tuning on videos. Therefore, we try to explore the collection of CoT datasets in videos to lead to video OpenQA and improve the reasoning ability of MLLMs. Unfortunately, making such video CoT datasets is not an easy task. Given that human annotation is too cumbersome and expensive, while machine-generated is not reliable due to the hallucination issue, we develop an automatic annotation tool that combines machine and human experts, under the active learning paradigm. Active learning is an interactive strategy between the model and human experts, in this way, the workload of human labeling can be reduced and the quality of the dataset can be guaranteed. With the help of the automatic annotation tool, we strive to contribute three datasets, namely VideoCoT, TopicQA, TopicCoT. Furthermore, we propose a simple but effective benchmark based on the collected datasets, which exploits CoT to maximize the complex reasoning capabilities of MLLMs. Extensive experiments demonstrate the effectiveness our solution, and we will release our source codes and datasets to facilitate the research community.