@inproceedings{alikhani-etal-2025-percor,
    title = "{P}er{C}o{R}: Evaluating Commonsense Reasoning in {P}ersian via Multiple-Choice Sentence Completion",
    author = "Alikhani, Morteza  and
      Bagherifard, Mohammadtaha  and
      Zinvandi, Erfan  and
      Sarmadi, Mehran",
    editor = "Inui, Kentaro  and
      Sakti, Sakriani  and
      Wang, Haofen  and
      Wong, Derek F.  and
      Bhattacharyya, Pushpak  and
      Banerjee, Biplab  and
      Ekbal, Asif  and
      Chakraborty, Tanmoy  and
      Singh, Dhirendra Pratap",
    booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.ijcnlp-long.120/",
    pages = "2205--2224",
    ISBN = "979-8-89176-298-5",
    abstract = "We introduced PerCoR{---}Persian Commonsense Reasoning{---}the first large-scale Persian benchmark for commonsense reasoning. PerCoR contains 106K multiple-choice sentence-completion problems drawn from more than forty news, cultural and other web sources. We adopt a linguistically grounded, conjunction-based segmentation strategy to generate coherent prefix{--}continuation pairs. To create challenging distractors, we propose DRESS-AF{---}Distractor Ranking via Embedding Similarity Scoring and Adversarial Filtering{---}a generation-free adversarial filtering method that selects distractors from the pool of gold continuations while maximising model confusion. Human annotators score 89{\%} on PerCoR, while OpenAI-o3 achieves the highest performance at 92.18{\%}, followed closely by Claude-Sonnet-3.7 (91.17{\%}). The strongest open-source model, DeepSeek-R1, reaches 82.51{\%}, underscoring both the dataset{'}s difficulty and the remaining performance gap in Persian commonsense reasoning. We further show that DRESS-AF transfers to the English HellaSwag benchmark, increasing its difficulty without hurting human solvability. The dataset is available at https://huggingface.co/datasets/MCINext/PerCoR ."
}