Multi-label Sequential Sentence Classification via Large Language Model

Mengfei Lan, Lecheng Zheng, Shufan Ming, Halil Kilicoglu

Abstract
Sequential sentence classification (SSC) in scientific publications is crucial for supporting downstream tasks such as fine-grained information retrieval and extractive summarization. However, current SSC methods are constrained by model size, sequence length, and single-label setting. To address these limitations, this paper proposes LLM-SSC, a large language model (LLM)-based framework for both single- and multi-label SSC tasks. Unlike previous approaches that employ small- or medium-sized language models, the proposed framework utilizes LLMs to generate SSC labels through designed prompts, which enhance task understanding by incorporating demonstrations and a query to describe the prediction target. We also present a multi-label contrastive learning loss with auto-weighting scheme, enabling the multi-label classification task. To support our multi-label SSC analysis, we introduce and release a new dataset, biorc800, which mainly contains unstructured abstracts in the biomedical domain with manual annotations. Experiments demonstrate LLM-SSC’s strong performance in SSC under both in-context learning and task-specific tuning settings. We release biorc800 and our code at: https://github.com/ScienceNLP-Lab/LLM-SSC.
Anthology ID:
2024.findings-emnlp.944
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16086–16104
Language:
URL:
https://aclanthology.org/2024.findings-emnlp.944
DOI:
10.18653/v1/2024.findings-emnlp.944
Bibkey:
Cite (ACL):
Mengfei Lan, Lecheng Zheng, Shufan Ming, and Halil Kilicoglu. 2024. Multi-label Sequential Sentence Classification via Large Language Model. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16086–16104, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Multi-label Sequential Sentence Classification via Large Language Model (Lan et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2024.findings-emnlp.944.pdf
PDF Search