Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation
Fei-Tzin Lee, Miguel Ballesteros, Feng Nan, Kathleen McKeown
Abstract
Large pretrained language models offer powerful generation capabilities, but cannot be reliably controlled at a sub-sentential level. We propose to make such fine-grained control possible in pretrained LMs by generating text directly from a semantic representation, Abstract Meaning Representation (AMR), which is augmented at the node level with syntactic control tags. We experiment with English-language generation of three modes of syntax relevant to the framing of a sentence - verb voice, verb tense, and realization of human entities - and demonstrate that they can be reliably controlled, even in settings that diverge drastically from the training distribution. These syntactic aspects contribute to how information is framed in text, something that is important for applications such as summarization which aim to highlight salient information.- Anthology ID:
- 2022.coling-1.514
- Volume:
- Proceedings of the 29th International Conference on Computational Linguistics
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Editors:
- Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, Seung-Hoon Na
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 5882–5895
- Language:
- URL:
- https://aclanthology.org/2022.coling-1.514
- DOI:
- Cite (ACL):
- Fei-Tzin Lee, Miguel Ballesteros, Feng Nan, and Kathleen McKeown. 2022. Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5882–5895, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
- Cite (Informal):
- Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation (Lee et al., COLING 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2022.coling-1.514.pdf
- Data
- AMR Bank