@inproceedings{bangash-etal-2025-musescorer,
    title = "{M}use{S}corer: Idea Originality Scoring At Scale",
    author = "Bangash, Ali Sarosh  and
      Veera, Krish  and
      Islam, Ishfat Abrar  and
      Baten, Raiyan Abdul",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-luhme/2025.emnlp-main.1009/",
    doi = "10.18653/v1/2025.emnlp-main.1009",
    pages = "19947--19965",
    ISBN = "979-8-89176-332-6",
    abstract = "An objective, face-valid method for scoring idea originality is to measure each idea{'}s statistical infrequency within a population{---}an approach long used in creativity research. Yet, computing these frequencies requires manually bucketing idea rephrasings, a process that is subjective, labor-intensive, error-prone, and brittle at scale. We introduce MuseScorer, a fully automated, psychometrically validated system for frequency-based originality scoring. MuseScorer integrates a Large Language Model (LLM) with externally orchestrated retrieval: given a new idea, it retrieves semantically similar prior idea-buckets and zero-shot prompts the LLM to judge whether the idea fits an existing bucket or forms a new one. These buckets enable frequency-based originality scoring without human annotation. Across five datasets ($N_{\text{participants}}=1143$, $n_{\text{ideas}}{=}16{,}294$), MuseScorer matches human annotators in idea clustering structure (AMI $=0.59$) and participant-level scoring ($r = 0.89$), while demonstrating strong convergent and external validity. The system enables scalable, intent-sensitive, and human-aligned originality assessment for creativity research."
}