Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks

Yingji Zhang, Danilo Carvalho, Andre Freitas

Abstract
Disentangled latent spaces usually have better semantic separability and geometrical properties, which leads to better interpretability and more controllable data generation. While this has been well investigated in Computer Vision, in tasks such as image disentanglement, in the NLP domain, sentence disentanglement is still comparatively under-investigated. Most previous work have concentrated on disentangling task-specific generative factors, such as sentiment, within the context of style transfer. In this work, we focus on a more general form of sentence disentanglement, targeting the localised modification and control of more general sentence semantic features. To achieve this, we contribute to a novel notion of sentence semantic disentanglement and introduce a flow-based invertible neural network (INN) mechanism integrated with a transformer-based language Autoencoder (AE) in order to deliver latent spaces with better separability properties. Experimental results demonstrate that the model can conform the distributed latent space into a better semantically disentangled sentence space, leading to improved language interpretability and controlled generation when compared to the recent state-of-the-art language VAE models.
Anthology ID:
2024.acl-long.116
Volume:
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2113–2134
Language:
URL:
https://aclanthology.org/2024.acl-long.116
DOI:
10.18653/v1/2024.acl-long.116
Bibkey:
Cite (ACL):
Yingji Zhang, Danilo Carvalho, and Andre Freitas. 2024. Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2113–2134, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks (Zhang et al., ACL 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/autopr/2024.acl-long.116.pdf
PDF Search Fix metadata