Abstract
This paper describes a new shared task for humor understanding that attempts to eschew the ubiquitous binary approach to humor detection and focus on comparative humor ranking instead. The task is based on a new dataset of funny tweets posted in response to shared hashtags, collected from the ‘Hashtag Wars’ segment of the TV show @midnight. The results are evaluated in two subtasks that require the participants to generate either the correct pairwise comparisons of tweets (subtask A), or the correct ranking of the tweets (subtask B) in terms of how funny they are. 7 teams participated in subtask A, and 5 teams participated in subtask B. The best accuracy in subtask A was 0.675. The best (lowest) rank edit distance for subtask B was 0.872.- Anthology ID:
- S17-2004
- Volume:
- Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)
- Month:
- August
- Year:
- 2017
- Address:
- Vancouver, Canada
- Editors:
- Steven Bethard, Marine Carpuat, Marianna Apidianaki, Saif M. Mohammad, Daniel Cer, David Jurgens
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 49–57
- Language:
- URL:
- https://aclanthology.org/S17-2004
- DOI:
- 10.18653/v1/S17-2004
- Cite (ACL):
- Peter Potash, Alexey Romanov, and Anna Rumshisky. 2017. SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 49–57, Vancouver, Canada. Association for Computational Linguistics.
- Cite (Informal):
- SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor (Potash et al., SemEval 2017)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/S17-2004.pdf