Abstract
Many recent advances in deep learning for natural language processing have come at increasing computational cost, but the power of these state-of-the-art models is not needed for every example in a dataset. We demonstrate two approaches to reducing unnecessary computation in cases where a fast but weak baseline classier and a stronger, slower model are both available. Applying an AUC-based metric to the task of sentiment classification, we find significant efficiency gains with both a probability-threshold method for reducing computational cost and one that uses a secondary decision network.- Anthology ID:
- W17-2631
- Volume:
- Proceedings of the 2nd Workshop on Representation Learning for NLP
- Month:
- August
- Year:
- 2017
- Address:
- Vancouver, Canada
- Editors:
- Phil Blunsom, Antoine Bordes, Kyunghyun Cho, Shay Cohen, Chris Dyer, Edward Grefenstette, Karl Moritz Hermann, Laura Rimell, Jason Weston, Scott Yih
- Venue:
- RepL4NLP
- SIG:
- SIGREP
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 257–264
- Language:
- URL:
- https://aclanthology.org/W17-2631
- DOI:
- 10.18653/v1/W17-2631
- Cite (ACL):
- Alexander Johansen and Richard Socher. 2017. Learning when to skim and when to read. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 257–264, Vancouver, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Learning when to skim and when to read (Johansen & Socher, RepL4NLP 2017)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/W17-2631.pdf
- Data
- SST