@inproceedings{eltanbouly-etal-2025-llms,
    title = "Can {LLM}s Directly Retrieve Passages for Answering Questions from Qur{'}an?",
    author = "Eltanbouly, Sohaila  and
      Albatarni, Salam  and
      Hassanein, Shaimaa  and
      Elsayed, Tamer",
    editor = "Darwish, Kareem  and
      Ali, Ahmed  and
      Abu Farha, Ibrahim  and
      Touileb, Samia  and
      Zitouni, Imed  and
      Abdelali, Ahmed  and
      Al-Ghamdi, Sharefah  and
      Alkhereyf, Sakhar  and
      Zaghouani, Wajdi  and
      Khalifa, Salam  and
      AlKhamissi, Badr  and
      Almatham, Rawan  and
      Hamed, Injy  and
      Alyafeai, Zaid  and
      Alowisheq, Areeb  and
      Inoue, Go  and
      Mrini, Khalil  and
      Alshammari, Waad",
    booktitle = "Proceedings of The Third Arabic Natural Language Processing Conference",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.arabicnlp-main.16/",
    pages = "203--210",
    ISBN = "979-8-89176-352-4",
    abstract = "The Holy Qur{'}an provides timeless guidance, addressing modern challenges and offering answers to many important questions. The Qur{'}an QA 2023 shared task introduced the Qur{'}anic Passage Retrieval (QPR) task, which involves retrieving relevant passages in response to MSA questions. In this work, we evaluate the ability of seven pre-trained large language models (LLMs) to retrieve relevant passages from the Qur{'}an in response to given questions, considering zero-shot and several few-shot scenarios. Our experiments show that the best model, Claude, significantly outperforms the state-of-the-art QPR model by 28 points on MAP and 38 points on MRR, exhibiting an impressive improvement of about 113{\%} and 82{\%}, respectively."
}