@article{braslavski-etal-2026-chronicles,
    title = "The Chronicles of {R}i{D}i{C}: Generating Datasets with Controlled Popularity Distribution for Long-form Factuality Evaluation",
    author = "Braslavski, Pavel  and
      Iarosh, Dmitrii  and
      Sushko, Nikita Sergeevich  and
      Sakhovskiy, Andrey  and
      Konovalov, Vasily  and
      Tutubalina, Elena  and
      Panchenko, Alexander",
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    journal = "International Conference on Language Resources and Evaluation",
    volume = "main",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://preview.aclanthology.org/ingest-lrec/2026.lrec-main.776/",
    pages = "9893--9904",
    abstract = "We present a configurable pipeline and the associated code that can be used to generate multilingual sets of entities with specified characteristics, such as domain, geographical location and popularity, using data from Wikipedia and Wikidata. These datasets are intended for evaluating the factuality of LLMs' long-form generation, thereby complementing evaluation based on short-form QA datasets. We present the RiDiC dataset as an example of this approach. RiDiC contains 3,000 entities from three domains {--} rivers, natural disasters, and car models {--} spanning different popularity tiers. Each entity is accompanied by its geographical location, English and Chinese names (if available) and relevant English and Chinese Wikipedia content, which is used to evaluate LLMs' responses. Generations about RiDiC entities were obtained from three LLMs in English and Chinese. These were then evaluated using a third-party factuality checker, which showed that entities from our dataset caused even frontier models to hallucinate. The code, data and generation/evaluation scripts have been released to enable the approach to be extended to new LLMs, languages and domains."
}