@inproceedings{ma-etal-2025-reasoning,
    title = "Reasoning or Memorization? Investigating {LLM}s' Capability in Restoring {C}hinese {I}nternet Homophones",
    author = "Ma, Jianfei  and
      Feng, Zhaoxin  and
      Song, Huacheng  and
      Chersoni, Emmanuele  and
      Chen, Zheng",
    editor = "Zhang, Yuji  and
      Chen, Canyu  and
      Li, Sha  and
      Geva, Mor  and
      Han, Chi  and
      Wang, Xiaozhi  and
      Feng, Shangbin  and
      Gao, Silin  and
      Augenstein, Isabelle  and
      Bansal, Mohit  and
      Li, Manling  and
      Ji, Heng",
    booktitle = "Proceedings of the 3rd Workshop on Towards Knowledgeable Foundation Models (KnowFM)",
    month = aug,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.knowllm-1.11/",
    pages = "120--139",
    ISBN = "979-8-89176-283-1",
    abstract = "Chinese homophones, prevalent in Internet culture, bring rich linguistic twists that are challenging for language models. While native speakers disambiguate them through phonological reasoning and contextual understanding, it remains untested how well LLMs perform on this task and whether LLMs also achieve this via similar reasoning processes or merely through memorization of homophone-original word pairs during training.In this paper, we present HomoP-CN, the first Chinese Internet homophones dataset with systematic perturbations for evaluating LLMs' homophone restoration capabilities. Using this benchmark, we investigated the influence of semantic, phonological, and graphemic features on LLMs' restoration accuracy, measured the reliance levels of each model on memorization during restoration through consistency ratios under controlled perturbations, and assessed the effectiveness of various prompting strategies, including contextual cues, pinyin augmentation, few-shot learning, and thought-chain approaches."
}