基于动态聚类与标签空间映射的上下文学习模板构建方法(In-Context Learning Demonstration Construction Method based on Dynamic Clustering and Label Space Mapping)
Zhang Qi (张琦), Jin Xingnan (金醒男), Pei Yu (裴誉), Du Yongping (杜永萍)
Abstract
“面向大语言模型提供自然语言指令,可生成预期输出,体现了其上下文学习能力。上下文学习的性能与上下文模板质量密切相关,现有的工作通常使用单一的选择算法进行模板构建,无法充分激发上下文学习能力。本文提出基于动态聚类与标签空间映射的上下文学习模板构建方法,动态选择相关示例,进一步提出聚类筛选方法,实现不同语义簇中示例多样化的选择。设计基于损失函数的排序选择方法,评估模板学习正确标签空间映射分布的能力,排序形成最终模板。在自然语言推理等任务中的实验结果表明,本文提出的方法使两个不同的大语言模型准确率最高分别提升3.2%和8.9%。”- Anthology ID:
- 2024.ccl-1.69
- Volume:
- Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 1: Main Conference)
- Month:
- July
- Year:
- 2024
- Address:
- Taiyuan, China
- Editors:
- Maosong Sun, Jiye Liang, Xianpei Han, Zhiyuan Liu, Yulan He
- Venue:
- CCL
- SIG:
- Publisher:
- Chinese Information Processing Society of China
- Note:
- Pages:
- 883–893
- Language:
- Chinese
- URL:
- https://preview.aclanthology.org/add-emnlp-2024-awards/2024.ccl-1.69/
- DOI:
- Cite (ACL):
- Zhang Qi, Jin Xingnan, Pei Yu, and Du Yongping. 2024. 基于动态聚类与标签空间映射的上下文学习模板构建方法(In-Context Learning Demonstration Construction Method based on Dynamic Clustering and Label Space Mapping). In Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 1: Main Conference), pages 883–893, Taiyuan, China. Chinese Information Processing Society of China.
- Cite (Informal):
- 基于动态聚类与标签空间映射的上下文学习模板构建方法(In-Context Learning Demonstration Construction Method based on Dynamic Clustering and Label Space Mapping) (Qi et al., CCL 2024)
- PDF:
- https://preview.aclanthology.org/add-emnlp-2024-awards/2024.ccl-1.69.pdf