K-hop neighbourhood regularization for few-shot learning on graphs: A case study of text classification
Niels van der Heijden, Ekaterina Shutova, Helen Yannakoudakis
Abstract
We present FewShotTextGCN, a novel method designed to effectively utilize the properties of word-document graphs for improved learning in low-resource settings. We introduce K-hop Neighbourhood Regularization, a regularizer for heterogeneous graphs, and show that it stabilizes and improves learning when only a few training samples are available. We furthermore propose a simplification in the graph-construction method, which results in a graph that is ∼7 times less dense and yields better performance in little-resource settings while performing on par with the state of the art in high-resource settings. Finally, we introduce a new variant of Adaptive Pseudo-Labeling tailored for word-document graphs. When using as little as 20 samples for training, we outperform a strong TextGCN baseline with 17% in absolute accuracy on average over eight languages. We demonstrate that our method can be applied to document classification without any language model pretraining on a wide range of typologically diverse languages while performing on par with large pretrained language models.- Anthology ID:
- 2023.eacl-main.85
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1187–1200
- Language:
- URL:
- https://aclanthology.org/2023.eacl-main.85
- DOI:
- 10.18653/v1/2023.eacl-main.85
- Cite (ACL):
- Niels van der Heijden, Ekaterina Shutova, and Helen Yannakoudakis. 2023. K-hop neighbourhood regularization for few-shot learning on graphs: A case study of text classification. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1187–1200, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- K-hop neighbourhood regularization for few-shot learning on graphs: A case study of text classification (van der Heijden et al., EACL 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.eacl-main.85.pdf