@inproceedings{huang-etal-2026-ritek,
    title = "{R}i{T}e{K}: A Dataset for Large Language Models Complex Reasoning over Textual Knowledge Graphs in Medicine",
    author = "Huang, Jiatan  and
      Li, Mingchen  and
      Yao, Zonghai  and
      Li, Dawei  and
      Zhang, Yuxin  and
      Yang, Zhichao  and
      Xiao, Yongkang  and
      Ouyang, Feiyun  and
      Li, Xiaohan  and
      Han, Shuo  and
      yu, Hong",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1793/",
    pages = "35985--36001",
    ISBN = "979-8-89176-395-1",
    abstract = "Answering complex real-world questions in the medical domain often requires accurate retrieval from medical Textual Knowledge Graphs (medical TKGs), as the relational path information from TKGs could enhance the inference ability of Large Language Models (LLMs). However, the main bottlenecks lie in the scarcity of existing medical TKGs, the limited expressiveness of their topological structures, and the lack of comprehensive evaluations of current retrievers for medical TKGs. To address these challenges, we first develop a dataset for LLMs Complex \textbf{R}eason\textbf{i}ng over medical \textbf{Te}xtual \textbf{K}nowledge Graphs (RiTeK), covering a broad range of topological structures. Specifically, we synthesize realistic user queries integrating diverse topological structures, relational information, and complex textual descriptions. We conduct a rigorous medical expert evaluation process to assess and validate the quality of our synthesized queries. RiTeK also serves as a comprehensive benchmark dataset for evaluating the capabilities of retrieval systems built upon LLMs. By assessing 11 representative retrievers on this benchmark, we observe that existing methods struggle to perform well, revealing notable limitations in current LLM-driven retrieval approaches. These findings highlight the pressing need for more effective retrieval systems tailored for semi-structured data in the medical domain."
}