Jiaxin Ai
2026
MeepleLM: A Virtual Playtester Simulating Diverse Subjective Experiences
Zizhen Li | Chuanhao Li | Yibin Wang | Jianwen Sun | Yukang Feng | Jiaxin Ai | Fanrui Zhang | Mingzhu Sun | Yifei Huang | Kaipeng Zhang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zizhen Li | Chuanhao Li | Yibin Wang | Jianwen Sun | Yukang Feng | Jiaxin Ai | Fanrui Zhang | Mingzhu Sun | Yifei Huang | Kaipeng Zhang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent advancements have expanded the role of Large Language Models (LLMs) in board games from playing agents to creative co-designers. However, a critical gap remains: current systems lack the capacity to offer constructive critique grounded in the emergent user experience. Bridging this gap is fundamental for harmonizing Human-AI collaboration, as it empowers designers to refine their creations via external perspectives while steering models away from biased or unpredictable outcomes. Automating this evaluation presents two challenges: inferring the latent dynamics connecting static rules to gameplay without an explicit engine, and modeling the subjective heterogeneity of diverse player groups. To address these, we curate a comprehensive dataset of 1,727 structurally corrected rulebooks and 150K reviews selected via rigorous quality scoring and facet-aware sampling. We augment this data with Mechanics-Dynamics-Aesthetics (MDA) reasoning to explicitly bridge the causal gap between written rules and player experience. We further distill distinct player personas and introduce MeepleLM, a specialized model that internalizes persona-specific reasoning patterns to accurately simulate the subjective feedback of diverse player archetypes. Extensive experiments demonstrate that MeepleLM significantly outperforms latest commercial models (e.g., GPT-5.1, Gemini3-Pro) in community alignment and critique quality, achieving a 70% preference rate in user studies assessing practical utility. MeepleLM serves as a reliable virtual playtester that provides experience-grounded feedback, offering a practical step towards audience-aligned Human-AI collaboration.
2025
MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification
xu Zhao Pan | Pengfei Zhou | Jiaxin Ai | Wangbo Zhao | Kai Wang | Xiaojiang Peng | Wenqi Shao | Hongxun Yao | Kaipeng Zhang
Findings of the Association for Computational Linguistics: ACL 2025
xu Zhao Pan | Pengfei Zhou | Jiaxin Ai | Wangbo Zhao | Kai Wang | Xiaojiang Peng | Wenqi Shao | Hongxun Yao | Kaipeng Zhang
Findings of the Association for Computational Linguistics: ACL 2025
Reasoning is an essential capacity for large language models (LLMs) to address complex tasks, whereas the identification of process errors is vital for improving this ability. Recently, process-level reward models (PRMs) were proposed to provide step-wise rewards that facilitate reinforcement learning and data production during training and guide LLMs toward correct steps during inference, thereby improving reasoning accuracy. However, existing benchmarks of PRMs are text-based and focus on error detection, neglecting other scenarios like reasoning search. To address this gap, we introduce MPBench, a comprehensive, multi-task, multimodal benchmark designed to systematically assess the effectiveness of PRMs in diverse scenarios. MPBench employs three evaluation paradigms, each targeting a specific role of PRMs in the reasoning process: (1) Step Correctness, which assesses the correctness of each intermediate reasoning step; (2) Answers Aggregation, which aggregates multiple solutions and selects the best one; and (3) Reasoning Process Search, which guides the search for optimal reasoning steps during inference. Through these paradigms, MPBench makes comprehensive evaluations and provides insights into the development of multimodal PRMs.
InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles
Zizhen Li | Chuanhao Li | Yibin Wang | Qi Chen | Diping Song | Yukang Feng | Jianwen Sun | Jiaxin Ai | Fanrui Zhang | Mingzhu Sun | Kaipeng Zhang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Zizhen Li | Chuanhao Li | Yibin Wang | Qi Chen | Diping Song | Yukang Feng | Jianwen Sun | Jiaxin Ai | Fanrui Zhang | Mingzhu Sun | Kaipeng Zhang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
LLMs have shown strong performance on human-centric reasoning tasks. While previous evaluations have explored whether LLMs can infer intentions or detect deception, they often overlook the individualized reasoning styles that influence how people interpret and act in social contexts. Social deduction games (SDGs) provide a natural testbed for evaluating individualized reasoning styles, where different players may adopt diverse but contextually valid reasoning strategies under identical conditions. To address this, we introduce InMind, a cognitively grounded evaluation framework designed to assess whether LLMs can capture and apply personalized reasoning styles in SDGs. InMind enhances structured gameplay data with round-level strategy traces and post-game reflections, collected under both Observer and Participant modes. It supports four cognitively motivated tasks that jointly evaluate both static alignment and dynamic adaptation. As a case study, we apply InMind to the game Avalon, evaluating 11 state-of-the-art LLMs. General-purpose LLMs, even GPT-4o frequently rely on lexical cues, struggling to anchor reflections in temporal gameplay or adapt to evolving strategies. In contrast, reasoning-enhanced LLMs like DeepSeek-R1 exhibit early signs of style-sensitive reasoning. These findings reveal key limitations in current LLMs’ capacity for individualized, adaptive reasoning, and position InMind as a step toward cognitively aligned human–AI interaction.