@inproceedings{arnold-2025-inspecting,
    title = "Inspecting the Representation Manifold of Differentially-Private Text",
    author = "Arnold, Stefan",
    editor = "Habernal, Ivan  and
      Ghanavati, Sepideh  and
      Jain, Vijayanta  and
      Igamberdiev, Timour  and
      Wilson, Shomir",
    booktitle = "Proceedings of the Sixth Workshop on Privacy in Natural Language Processing",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.privatenlp-main.5/",
    pages = "53--59",
    ISBN = "979-8-89176-246-6",
    abstract = "Differential Privacy (DP) for text has recently taken the form of text paraphrasing using language models and temperature sampling to better balance privacy and utility. However, the geometric distortion of DP regarding the structure and complexity in the representation space remains unexplored. By estimating the intrinsic dimension of paraphrased text across varying privacy budgets, we find that word-level methods severely raise the representation manifold, while sentence-level methods produce paraphrases whose manifolds are topologically more consistent with human-written paraphrases. Among sentence-level methods, masked paraphrasing, compared to causal paraphrasing, demonstrates superior preservation of structural complexity, suggesting that autoregressive generation propagates distortions from unnatural word choices that cascade and inflate the representation space."
}