Zhonghao Li


2025

pdf bib
StructFact: Reasoning Factual Knowledge from Structured Data with Large Language Models
Sirui Huang | Yanggan Gu | Zhonghao Li | Xuming Hu | Li Qing | Guandong Xu
Findings of the Association for Computational Linguistics: ACL 2025

Large language models (LLMs) have made significant strides in natural language processing by leveraging their ability to comprehend and reason with factual knowledge. However, a significant amount of factual knowledge is stored in structured data, which has unique characteristics not typically encountered in the unstructured texts used for pretraining LLMs. To evaluate the capability of LLMs in handling facts structurally stored, we introduce a benchmark called StructFact, which includes meticulously annotated factual questions, spanning five tasks that reflect the intrinsic properties of structured data. This benchmark aims to delineate the strengths and limitations of LLMs in reasoning with structured data for knowledge-intensive tasks in practical applications. Extensive experiments conducted on 10 common LLMs have yielded several insights, one notable finding being that these models struggle significantly with the heterogeneity of structured data during reasoning.

2024

pdf bib
Refiner: Restructure Retrieved Content Efficiently to Advance Question-Answering Capabilities
Zhonghao Li | Xuming Hu | Aiwei Liu | Kening Zheng | Sirui Huang | Hui Xiong
Findings of the Association for Computational Linguistics: EMNLP 2024