Brian Waltman

Fixing paper assignments

  1. Please select all papers that belong to the same person.
  2. Indicate below which author they should be assigned to.
Provide a valid ORCID iD here. This will be used to match future papers to this author.
Provide the name of the school or the university where the author has received or will receive their highest degree (e.g., Ph.D. institution for researchers, or current affiliation for students). This will be used to form the new author page ID, if needed.

TODO: "submit" and "cancel" buttons here

2025

pdf bib
Bias and Reliability in AI Safety Assessment: Multi-Facet Rasch Analysis of Human Moderators
Chunling Niu | Kelly Bradley | Biao Ma | Brian Waltman | Loren Cossette | Rui Jin
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers

Using Multi-Facet Rasch Modeling on 36,400 safety ratings of AI-generated conversations, we reveal significant racial disparities (Asian 39.1%, White 28.7% detection rates) and content-specific bias patterns. Simulations show that diverse teams of 8-10 members achieve 70%+ reliability versus 62% for smaller homogeneous teams, providing evidence-based guidelines for AI-generated content moderation.
Search
Co-authors
Venues
Fix data