Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymization
Gabriel Loiseau, Damien Sileo, Damien Riquet, Maxime Meyer, Marc Tommasi
Abstract
Text anonymization is the process of removing or obfuscating information from textual data to protect the privacy of individuals. This process inherently involves a complex trade-off between privacy protection and information preservation, where stringent anonymization methods can significantly impact the text’s utility for downstream applications. Evaluating the effectiveness of text anonymization proves challenging from both privacy and utility perspectives, as there is no universal benchmark that can comprehensively assess anonymization techniques across diverse, and sometimes contradictory contexts. We present Tau-Eval, an open-source framework for benchmarking text anonymization methods through the lens of privacy and utility task sensitivity. A Python library, code, documentation and tutorials are publicly available.- Anthology ID:
- 2025.emnlp-demos.16
- Volume:
- Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
- Month:
- November
- Year:
- 2025
- Address:
- Suzhou, China
- Editors:
- Ivan Habernal, Peter Schulam, Jörg Tiedemann
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 216–227
- Language:
- URL:
- https://preview.aclanthology.org/author-page-zheyuan-zhang-jhu/2025.emnlp-demos.16/
- DOI:
- 10.18653/v1/2025.emnlp-demos.16
- Cite (ACL):
- Gabriel Loiseau, Damien Sileo, Damien Riquet, Maxime Meyer, and Marc Tommasi. 2025. Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 216–227, Suzhou, China. Association for Computational Linguistics.
- Cite (Informal):
- Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymization (Loiseau et al., EMNLP 2025)
- PDF:
- https://preview.aclanthology.org/author-page-zheyuan-zhang-jhu/2025.emnlp-demos.16.pdf