Abstract
Multi-label text classification (MLTC) aims to assign multiple labels to a given text. Previous works have focused on text representation learning and label correlations modeling using pre-trained language models (PLMs). However, studies have shown that PLMs generate word frequency-oriented text representations, causing texts with different labels to be closely distributed in a narrow region, which is difficult to classify. To address this, we present a novel framework CL( ̲Contrastive ̲Learning)-MIL ( ̲Multi-granularity ̲Information ̲Learning) to refine the text representation for MLTC task. We first use contrastive learning to generate uniform initial text representation and incorporate label frequency implicitly. Then, we design a multi-task learning module to integrate multi-granularity (diverse text-labels correlations, label-label relations and label frequency) information into text representations, enhancing their discriminative ability. Experimental results demonstrate the complementarity of the modules in CL-MIL, improving the quality of text representations and yielding stable and competitive improvements for MLTC.- Anthology ID:
- 2023.findings-emnlp.635
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 9470–9480
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.635
- DOI:
- 10.18653/v1/2023.findings-emnlp.635
- Cite (ACL):
- Fangfang Li, Puzhen Su, Junwen Duan, and Weidong Xiao. 2023. Towards Better Representations for Multi-Label Text Classification with Multi-granularity Information. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9470–9480, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Towards Better Representations for Multi-Label Text Classification with Multi-granularity Information (Li et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.findings-emnlp.635.pdf