@inproceedings{halim-etal-2025-jury,
    title = "Let The Jury Decide: Fair Demonstration Selection for In-Context Learning through Incremental Greedy Evaluation",
    author = "Halim, Sadaf Md  and
      Zhao, Chen  and
      Wu, Xintao  and
      Khan, Latifur  and
      Grant, Christan  and
      Rahman, Fariha Ishrat  and
      Chen, Feng",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/transition-to-people-yaml/2025.findings-acl.968/",
    doi = "10.18653/v1/2025.findings-acl.968",
    pages = "18914--18931",
    ISBN = "979-8-89176-256-5",
    abstract = "Large Language Models (LLMs) are powerful in-context learners, achieving strong performance with just a few high-quality demonstrations. However, fairness concerns arise in many in-context classification tasks, especially when predictions involve sensitive attributes. To address this, we propose JUDGE{---}a simple yet effective framework for selecting fair and representative demonstrations that improve group fairness in In-Context Learning. JUDGE constructs the demonstration set iteratively using a greedy approach, guided by a small, carefully selected jury set. Our method remains robust across varying LLM architectures and datasets, ensuring consistent fairness improvements. We evaluate JUDGE on four datasets using four LLMs, comparing it against seven baselines. Results show that JUDGE consistently improves fairness metrics without compromising accuracy."
}