Aspect-based summarization of pros and cons in unstructured product reviews
Florian Kunneman, Sander Wubben, Antal van den Bosch, Emiel Krahmer
Abstract
We developed three systems for generating pros and cons summaries of product reviews. Automating this task eases the writing of product reviews, and offers readers quick access to the most important information. We compared SynPat, a system based on syntactic phrases selected on the basis of valence scores, against a neural-network-based system trained to map bag-of-words representations of reviews directly to pros and cons, and the same neural system trained on clusters of word-embedding encodings of similar pros and cons. We evaluated the systems in two ways: first on held-out reviews with gold-standard pros and cons, and second by asking human annotators to rate the systems’ output on relevance and completeness. In the second evaluation, the gold-standard pros and cons were assessed along with the system output. We find that the human-generated summaries are not deemed as significantly more relevant or complete than the SynPat systems; the latter are scored higher than the human-generated summaries on a precision metric. The neural approaches yield a lower performance in the human assessment, and are outperformed by the baseline.- Anthology ID:
- C18-1188
- Volume:
- Proceedings of the 27th International Conference on Computational Linguistics
- Month:
- August
- Year:
- 2018
- Address:
- Santa Fe, New Mexico, USA
- Editors:
- Emily M. Bender, Leon Derczynski, Pierre Isabelle
- Venue:
- COLING
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2219–2229
- Language:
- URL:
- https://aclanthology.org/C18-1188
- DOI:
- Cite (ACL):
- Florian Kunneman, Sander Wubben, Antal van den Bosch, and Emiel Krahmer. 2018. Aspect-based summarization of pros and cons in unstructured product reviews. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2219–2229, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
- Cite (Informal):
- Aspect-based summarization of pros and cons in unstructured product reviews (Kunneman et al., COLING 2018)
- PDF:
- https://preview.aclanthology.org/naacl24-info/C18-1188.pdf
- Code
- fkunneman/Product_review_summary