Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks
Mrinmaya Sachan, Avinava Dubey, Eduard H. Hovy, Tom M. Mitchell, Dan Roth, Eric P. Xing
Abstract
To ensure readability, text is often written and presented with due formatting. These text formatting devices help the writer to effectively convey the narrative. At the same time, these help the readers pick up the structure of the discourse and comprehend the conveyed information. There have been a number of linguistic theories on discourse structure of text. However, these theories only consider unformatted text. Multimedia text contains rich formatting features that can be leveraged for various NLP tasks. In this article, we study some of these discourse features in multimedia text and what communicative function they fulfill in the context. As a case study, we use these features to harvest structured subject knowledge of geometry from textbooks. We conclude that the discourse and text layout features provide information that is complementary to lexical semantic information. Finally, we show that the harvested structured knowledge can be used to improve an existing solver for geometry problems, making it more accurate as well as more explainable.- Anthology ID:
- J19-4002
- Volume:
- Computational Linguistics, Volume 45, Issue 4 - December 2019
- Month:
- December
- Year:
- 2019
- Address:
- Cambridge, MA
- Venue:
- CL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 627–665
- Language:
- URL:
- https://aclanthology.org/J19-4002
- DOI:
- 10.1162/coli_a_00360
- Cite (ACL):
- Mrinmaya Sachan, Avinava Dubey, Eduard H. Hovy, Tom M. Mitchell, Dan Roth, and Eric P. Xing. 2019. Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks. Computational Linguistics, 45(4):627–665.
- Cite (Informal):
- Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks (Sachan et al., CL 2019)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/J19-4002.pdf