Abstract
In this paper, we assess the ability of BERT and its derivative models (RoBERTa, DistilBERT, and ALBERT) for short-edits based humor grading. We test these models for humor grading and classification tasks on the Humicroedit and the FunLines dataset. We perform extensive experiments with these models to test their language modeling and generalization abilities via zero-shot inference and cross-dataset inference based approaches. Further, we also inspect the role of self-attention layers in humor-grading by performing a qualitative analysis over the self-attention weights from the final layer of the trained BERT model. Our experiments show that all the pre-trained BERT derivative models show significant generalization capabilities for humor-grading related tasks.- Anthology ID:
- 2020.semeval-1.108
- Volume:
- Proceedings of the Fourteenth Workshop on Semantic Evaluation
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona (online)
- Editors:
- Aurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- International Committee for Computational Linguistics
- Note:
- Pages:
- 858–864
- Language:
- URL:
- https://aclanthology.org/2020.semeval-1.108
- DOI:
- 10.18653/v1/2020.semeval-1.108
- Cite (ACL):
- Siddhant Mahurkar and Rajaswa Patil. 2020. LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits Based Humor Grading. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 858–864, Barcelona (online). International Committee for Computational Linguistics.
- Cite (Informal):
- LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits Based Humor Grading (Mahurkar & Patil, SemEval 2020)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2020.semeval-1.108.pdf
- Data
- Humicroedit