Finding Convincing Arguments Using Scalable Bayesian Preference Learning

Edwin Simpson, Iryna Gurevych


Abstract
We introduce a scalable Bayesian preference learning method for identifying convincing arguments in the absence of gold-standard ratings or rankings. In contrast to previous work, we avoid the need for separate methods to perform quality control on training data, predict rankings and perform pairwise classification. Bayesian approaches are an effective solution when faced with sparse or noisy training data, but have not previously been used to identify convincing arguments. One issue is scalability, which we address by developing a stochastic variational inference method for Gaussian process (GP) preference learning. We show how our method can be applied to predict argument convincingness from crowdsourced data, outperforming the previous state-of-the-art, particularly when trained with small amounts of unreliable data. We demonstrate how the Bayesian approach enables more effective active learning, thereby reducing the amount of data required to identify convincing arguments for new users and domains. While word embeddings are principally used with neural networks, our results show that word embeddings in combination with linguistic features also benefit GPs when predicting argument convincingness.
Anthology ID:
Q18-1026
Volume:
Transactions of the Association for Computational Linguistics, Volume 6
Month:
Year:
2018
Address:
Cambridge, MA
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
357–371
Language:
URL:
https://aclanthology.org/Q18-1026
DOI:
10.1162/tacl_a_00026
Bibkey:
Cite (ACL):
Edwin Simpson and Iryna Gurevych. 2018. Finding Convincing Arguments Using Scalable Bayesian Preference Learning. Transactions of the Association for Computational Linguistics, 6:357–371.
Cite (Informal):
Finding Convincing Arguments Using Scalable Bayesian Preference Learning (Simpson & Gurevych, TACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/Q18-1026.pdf
Video:
 https://vimeo.com/285800680
Code
 UKPLab/tacl2018-preference-convincing