Sen Liu
2026
GQLBench: A Large-Scale Cross-Domain, Cross-Dialect Benchmark for NL2GQL
Yanning Su | Yuhang Zhou | Yang Fang | Sen Liu | Guangnan Ye | Hongfeng Chai
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yanning Su | Yuhang Zhou | Yang Fang | Sen Liu | Guangnan Ye | Hongfeng Chai
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Despite growing interest in NL2GQL, benchmarking progress has been constrained by the lack of resources that are simultaneously large-scale, cross-domain, and cross-dialect. To address this gap, we present **GQLBench**, a new benchmark built through an automated and scalable framework that integrates NL2SQL-to-NL2GQL conversion with graph-native data generation. GQLBench supports execution-based evaluation on both Cypher and ISO-GQL, covering hundreds of graph databases and over 20k natural language questions for each dialect. By combining converted data from mature NL2SQL resources with synthetic graph-specific queries, it captures both schema diversity from real-world relational sources and graph-native reasoning challenges, including long paths and cycles. Beyond overall performance comparison, GQLBench also enables fine-grained evaluation across dialects, graph patterns, and query complexity. Experiments on advanced LLMs show that even strong proprietary models struggle on GQLBench, with gemini-3-flash achieving only 35.40% average execution accuracy across the two dialects. Our data and code are available at https://github.com/qxssadf/GQLBench.
2025
Are LLMs Rational Investors? A Study on the Financial Bias in LLMs
Yuhang Zhou | Yuchen Ni | Zhiheng Xi | Zhangyue Yin | Yu He | Gan Yunhui | Xiang Liu | Zhang Jian | Sen Liu | Xipeng Qiu | Yixin Cao | Guangnan Ye | Hongfeng Chai
Findings of the Association for Computational Linguistics: ACL 2025
Yuhang Zhou | Yuchen Ni | Zhiheng Xi | Zhangyue Yin | Yu He | Gan Yunhui | Xiang Liu | Zhang Jian | Sen Liu | Xipeng Qiu | Yixin Cao | Guangnan Ye | Hongfeng Chai
Findings of the Association for Computational Linguistics: ACL 2025
Large language models (LLMs) excel in natural language generation but also exhibit biases, particularly in gender, race, and religion, which can be amplified with widespread use. However, research on biases in specific domains, such as finance, remains limited. To address this gap, we conducted a comprehensive evaluation of 23 leading LLMs and found varying degrees of financial bias, including more pronounced biases in financial-specific LLMs (FinLLMs). In response, we propose the Financial Bias Indicators (FBI) framework, which includes components like the Bias Unveiler, Bias Detective, Bias Tracker, and Bias Antidote, designed to identify, detect, analyze, and mitigate financial biases. Our analysis explores the root causes of these biases and introduces a debiasing method based on financial causal knowledge, alongside three other debiasing techniques. For the most biased model, we successfully reduced bias by 68% according to key metrics. This study advances our understanding of LLM biases in finance and highlights the need for greater scrutiny in their application within this critical domain.
2024
R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQL
Yuhang Zhou | Yu He | Siyu Tian | Yuchen Ni | Zhangyue Yin | Xiang Liu | Chuanjun Ji | Sen Liu | Xipeng Qiu | Guangnan Ye | Hongfeng Chai
Findings of the Association for Computational Linguistics: EMNLP 2024
Yuhang Zhou | Yu He | Siyu Tian | Yuchen Ni | Zhangyue Yin | Xiang Liu | Chuanjun Ji | Sen Liu | Xipeng Qiu | Guangnan Ye | Hongfeng Chai
Findings of the Association for Computational Linguistics: EMNLP 2024
While current tasks of converting natural language to SQL (NL2SQL) using Foundation Models have shown impressive achievements, adapting these approaches for converting natural language to Graph Query Language (NL2GQL) encounters hurdles due to the distinct nature of GQL compared to SQL, alongside the diverse forms of GQL. Moving away from traditional rule-based and slot-filling methodologies, we introduce a novel approach, R3-NL2GQL, integrating both small and large Foundation Models for ranking, rewriting, and refining tasks. This method leverages the interpretative strengths of smaller models for initial ranking and rewriting stages, while capitalizing on the superior generalization and query generation prowess of larger models for the final transformation of natural language queries into GQL formats. Addressing the scarcity of datasets in this emerging field, we have developed a bilingual dataset, sourced from graph database manuals and selected open-source Knowledge Graphs (KGs). Our evaluation of this methodology on this dataset demonstrates its promising efficacy and robustness.