NLP Systems That Can’t Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps

Kristina Gligoric, Myra Cheng, Lucia Zheng, Esin Durmus, Dan Jurafsky

Abstract
The use of words to convey speaker’s intent is traditionally distinguished from the ‘mention’ of words for quoting what someone said, or pointing out properties of a word. Here we show that computationally modeling this use-mention distinction is crucial for dealing with counterspeech online. Counterspeech that refutes problematic content often mentions harmful language but is not harmful itself (e.g., calling a vaccine dangerous is not the same as expressing disapproval of someone for calling vaccines dangerous). We show that even recent language models fail at distinguishing use from mention, and that this failure propagates to two key downstream tasks: misinformation and hate speech detection, resulting in censorship of counterspeech. We introduce prompting mitigations that teach the use-mention distinction, and show they reduce these errors. Our work highlights the importance of the use-mention distinction for NLP and CSS and offers ways to address it.
Anthology ID:
2024.naacl-long.331
Volume:
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5942–5959
Language:
URL:
https://preview.aclanthology.org/add_missing_videos/2024.naacl-long.331/
DOI:
10.18653/v1/2024.naacl-long.331
Bibkey:
Cite (ACL):
Kristina Gligoric, Myra Cheng, Lucia Zheng, Esin Durmus, and Dan Jurafsky. 2024. NLP Systems That Can’t Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5942–5959, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
NLP Systems That Can’t Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps (Gligoric et al., NAACL 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/add_missing_videos/2024.naacl-long.331.pdf
PDF Search Fix data