@inproceedings{zhang-etal-2025-quasi,
    title = "Quasi-symbolic Semantic Geometry over Transformer-based Variational {A}uto{E}ncoder",
    author = "Zhang, Yingji  and
      Carvalho, Danilo  and
      Freitas, Andre",
    editor = "Boleda, Gemma  and
      Roth, Michael",
    booktitle = "Proceedings of the 29th Conference on Computational Natural Language Learning",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/acl25-workshop-ingestion/2025.conll-1.2/",
    pages = "12--29",
    ISBN = "979-8-89176-271-8",
    abstract = "Formal/symbolic semantics can provide canonical, rigid controllability and interpretability to sentence representations due to their \textit{localisation} or \textit{composition} property. How can we deliver such property to the current distributional sentence representations to better control and interpret the generation of language models (LMs)? In this work, we theoretically frame the sentence semantics as the composition of \textit{semantic role - word content} features and propose the formal semantic geometrical framework. To inject such geometry into Transformer-based LMs (i.e. GPT2), we deploy a supervised Transformer-based Variational AutoEncoder, where the sentence generation can be manipulated and explained over low-dimensional latent Gaussian space. In addition, we propose a new probing algorithm to guide the movement of sentence vectors over such geometry. Experimental results reveal that the formal semantic geometry can potentially deliver better control and interpretation to sentence generation."
}