OtterlyObsessedWithSemantics at SemEval-2024 Task 4: Developing a Hierarchical Multi-Label Classification Head for Large Language Models
Julia Wunderle, Julian Schubert, Antonella Cacciatore, Albin Zehe, Jan Pfister, Andreas Hotho
Abstract
For our submission for Subtask 1, we developed a custom classification head that is designed to be applied atop of a Large Language Model. We reconstructed the hierarchy across multiple fully connected layers, allowing us to incorporate previous foundational decisions in subsequent, more fine-grained layers. To find the best hyperparameters, we conducted a grid-search and to compete in the multilingual setting, we translated all documents to English.- Anthology ID:
- 2024.semeval-1.90
- Volume:
- Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 602–612
- Language:
- URL:
- https://preview.aclanthology.org/add_missing_videos/2024.semeval-1.90/
- DOI:
- 10.18653/v1/2024.semeval-1.90
- Cite (ACL):
- Julia Wunderle, Julian Schubert, Antonella Cacciatore, Albin Zehe, Jan Pfister, and Andreas Hotho. 2024. OtterlyObsessedWithSemantics at SemEval-2024 Task 4: Developing a Hierarchical Multi-Label Classification Head for Large Language Models. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 602–612, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- OtterlyObsessedWithSemantics at SemEval-2024 Task 4: Developing a Hierarchical Multi-Label Classification Head for Large Language Models (Wunderle et al., SemEval 2024)
- PDF:
- https://preview.aclanthology.org/add_missing_videos/2024.semeval-1.90.pdf