Grouped-Attention for Content-Selection and Content-Plan Generation
Bayu Distiawan Trisedya, Xiaojie Wang, Jianzhong Qi, Rui Zhang, Qingjun Cui
Abstract
Content-planning is an essential part of data-to-text generation to determine the order of data mentioned in generated texts. Recent neural data-to-text generation models employ Pointer Networks to explicitly learn content-plan given a set of attributes as input. They use LSTM to encode the input, which assumes a sequential relationship in the input. This may be sub-optimal to encode a set of attributes, where the attributes have a composite structure: the attributes are disordered while each attribute value is an ordered list of tokens. We handle this problem by proposing a neural content-planner that can capture both local and global contexts of such a structure. Specifically, we propose a novel attention mechanism called GSC-attention. A key component of the GSC-attention is grouped-attention, which is token-level attention constrained within each input attribute that enables our proposed model captures both local and global context. Moreover, our content-planner explicitly learns content-selection, which is integrated into the content-planner to select the most important data to be included in the generated text via an attention masking procedure. Experimental results show that our model outperforms the competitors by 4.92%, 4.70%, and 16.56% in terms of Damerau-Levenshtein Distance scores on three real-world datasets.- Anthology ID:
- 2021.findings-emnlp.166
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2021
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Editors:
- Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
- Venue:
- Findings
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1935–1944
- Language:
- URL:
- https://aclanthology.org/2021.findings-emnlp.166
- DOI:
- 10.18653/v1/2021.findings-emnlp.166
- Cite (ACL):
- Bayu Distiawan Trisedya, Xiaojie Wang, Jianzhong Qi, Rui Zhang, and Qingjun Cui. 2021. Grouped-Attention for Content-Selection and Content-Plan Generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1935–1944, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Grouped-Attention for Content-Selection and Content-Plan Generation (Trisedya et al., Findings 2021)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2021.findings-emnlp.166.pdf
- Data
- RotoWire, WikiBio