Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation

Fei-Tzin Lee, Miguel Ballesteros, Feng Nan, Kathleen McKeown

Abstract
Large pretrained language models offer powerful generation capabilities, but cannot be reliably controlled at a sub-sentential level. We propose to make such fine-grained control possible in pretrained LMs by generating text directly from a semantic representation, Abstract Meaning Representation (AMR), which is augmented at the node level with syntactic control tags. We experiment with English-language generation of three modes of syntax relevant to the framing of a sentence - verb voice, verb tense, and realization of human entities - and demonstrate that they can be reliably controlled, even in settings that diverge drastically from the training distribution. These syntactic aspects contribute to how information is framed in text, something that is important for applications such as summarization which aim to highlight salient information.
Anthology ID:
2022.coling-1.514
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
5882–5895
Language:
URL:
https://aclanthology.org/2022.coling-1.514
DOI:
Bibkey:
Cite (ACL):
Fei-Tzin Lee, Miguel Ballesteros, Feng Nan, and Kathleen McKeown. 2022. Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5882–5895, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation (Lee et al., COLING 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.coling-1.514.pdf
Data
AMR Bank
PDF Search