Xuefeng Li
2022
PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling
Guanting Dong
|
Daichi Guo
|
Liwen Wang
|
Xuefeng Li
|
Zechen Wang
|
Chen Zeng
|
Keqing He
|
Jinzheng Zhao
|
Hao Lei
|
Xinyue Cui
|
Yi Huang
|
Junlan Feng
|
Weiran Xu
Proceedings of the 29th International Conference on Computational Linguistics
Most existing slot filling models tend to memorize inherent patterns of entities and corresponding contexts from training data. However, these models can lead to system failure or undesirable outputs when being exposed to spoken language perturbation or variation in practice. We propose a perturbed semantic structure awareness transferring method for training perturbation-robust slot filling models. Specifically, we introduce two MLM-based training strategies to respectively learn contextual semantic structure and word distribution from unsupervised language perturbation corpus. Then, we transfer semantic knowledge learned from upstream training procedure into the original samples and filter generated data by consistency processing. These procedures aims to enhance the robustness of slot filling models. Experimental results show that our method consistently outperforms the previous basic methods and gains strong generalization while preventing the model from memorizing inherent patterns of entities and contexts.
2021
Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling
Liwen Wang
|
Xuefeng Li
|
Jiachi Liu
|
Keqing He
|
Yuanmeng Yan
|
Weiran Xu
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Zero-shot cross-domain slot filling alleviates the data dependence in the case of data scarcity in the target domain, which has aroused extensive research. However, as most of the existing methods do not achieve effective knowledge transfer to the target domain, they just fit the distribution of the seen slot and show poor performance on unseen slot in the target domain. To solve this, we propose a novel approach based on prototypical contrastive learning with a dynamic label confusion strategy for zero-shot slot filling. The prototypical contrastive learning aims to reconstruct the semantic constraints of labels, and we introduce the label confusion strategy to establish the label dependence between the source domains and the target domain on-the-fly. Experimental results show that our model achieves significant improvement on the unseen slots, while also set new state-of-the-arts on slot filling task.
Search
Co-authors
- Liwen Wang 2
- Keqing He 2
- Weiran Xu 2
- Jiachi Liu 1
- Yuanmeng Yan 1
- show all...