A Simple Geometric Method for Cross-Lingual Linguistic Transformations with Pre-trained Autoencoders

Maarten De Raedt, Fréderic Godin, Pieter Buteneers, Chris Develder, Thomas Demeester

Abstract
Powerful sentence encoders trained for multiple languages are on the rise. These systems are capable of embedding a wide range of linguistic properties into vector representations. While explicit probing tasks can be used to verify the presence of specific linguistic properties, it is unclear whether the vector representations can be manipulated to indirectly steer such properties. For efficient learning, we investigate the use of a geometric mapping in embedding space to transform linguistic properties, without any tuning of the pre-trained sentence encoder or decoder. We validate our approach on three linguistic properties using a pre-trained multilingual autoencoder and analyze the results in both monolingual and cross-lingual settings.
Anthology ID:
2021.emnlp-main.792
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10108–10114
Language:
URL:
https://aclanthology.org/2021.emnlp-main.792
DOI:
10.18653/v1/2021.emnlp-main.792
Bibkey:
Cite (ACL):
Maarten De Raedt, Fréderic Godin, Pieter Buteneers, Chris Develder, and Thomas Demeester. 2021. A Simple Geometric Method for Cross-Lingual Linguistic Transformations with Pre-trained Autoencoders. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10108–10114, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
A Simple Geometric Method for Cross-Lingual Linguistic Transformations with Pre-trained Autoencoders (De Raedt et al., EMNLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/paclic-22-ingestion/2021.emnlp-main.792.pdf
Video:
 https://preview.aclanthology.org/paclic-22-ingestion/2021.emnlp-main.792.mp4
Data
SentEval
PDF Search Video