@inproceedings{zeinalipour-etal-2025-farsense,
    title = "{F}ar{S}ense: A Comprehensive Commonsense Benchmark and Evaluation Framework for the {F}arsi Language",
    author = "Zeinalipour, Kamyar  and
      Jamshidi, Neda  and
      Hejazi, Seyedehbahareh  and
      Maggini, Marco  and
      Bianchini, Monica  and
      Paoletti, Simone  and
      Gori, Marco",
    editor = "Inui, Kentaro  and
      Sakti, Sakriani  and
      Wang, Haofen  and
      Wong, Derek F.  and
      Bhattacharyya, Pushpak  and
      Banerjee, Biplab  and
      Ekbal, Asif  and
      Chakraborty, Tanmoy  and
      Singh, Dhirendra Pratap",
    booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.ijcnlp-long.187/",
    pages = "3529--3599",
    ISBN = "979-8-89176-298-5",
    abstract = "Although Farsi is widely spoken, no comprehensive benchmark exists for assessing commonsense reasoning in language models. We therefore present \textbf{FarSense}, a 6{-}task benchmark for Farsi covering True/False judgment, multiple-choice questions, Explanation, Cause{-}Effect inference, Counterfactual reasoning, and Knowledge Completion. Starting from Farsi{-}Wikipedia, we filtered noise and retained {\textasciitilde}4,210 passages, rewrote them into realistic daily scenarios, and derived the above tasks from each scenario. Scenario and task generation quality was first judged via native{-}speaker annotations on outputs from five major LLMs{---}GPT{-}4o, Gemini-2.5-Flash, Mistral-Large, Qwen{-}Plus, and DeepSeek{-}Chat. Gemini-2.5-Flash demonstrated the highest performance, leading to its use in generating a large-scale dataset, subsequently finalized through meticulous two-step human validation. Using \textbf{FarSense}, we measured the commonsense ability of the same five flagship LLMs and also fine{-}tuned six compact models (1B{--}24B parameters) before re{-}evaluating them. To ensure broad applicability, task wording was designed to minimize dialectal, cultural, or religious bias. Experiments show that targeted fine{-}tuning yields substantial gains, confirming \textbf{FarSense} as a reliable, openly licensed resource for advancing reproducible commonsense understanding research in Farsi NLP. We publicly release all code and data at https://github.com/KamyarZeinalipour/FarSense."
}