Tiffany Knearem
2025
ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
Hua Shen
|
Tiffany Knearem
|
Reshmi Ghosh
|
Yu-Ju Yang
|
Nicholas Clark
|
Tanu Mitra
|
Yun Huang
Proceedings of the 9th Widening NLP Workshop
As AI advances, aligning it with diverse human and societal values grows critical. But how do we define these values and measure AI’s adherence to them? We present ValueCompass, a framework grounded in psychological theories, to assess human-AI alignment. Applying it to five diverse LLMs and 112 humans from seven countries across four scenarios—collaborative writing, education, public sectors, and healthcare—we uncover key misalignments. For example, humans prioritize national security, while LLMs often reject it. Values also shift across contexts, demanding scenario-specific alignment strategies. This work advances AI design by mapping how systems can better reflect societal ethics.
Search
Fix author
Co-authors
- Nicholas Clark 1
- Reshmi Ghosh 1
- Yun Huang 1
- Tanu Mitra 1
- Hua Shen 1
- show all...