Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks

Mrinmaya Sachan, Avinava Dubey, Eduard H. Hovy, Tom M. Mitchell, Dan Roth, Eric P. Xing


Abstract
To ensure readability, text is often written and presented with due formatting. These text formatting devices help the writer to effectively convey the narrative. At the same time, these help the readers pick up the structure of the discourse and comprehend the conveyed information. There have been a number of linguistic theories on discourse structure of text. However, these theories only consider unformatted text. Multimedia text contains rich formatting features that can be leveraged for various NLP tasks. In this article, we study some of these discourse features in multimedia text and what communicative function they fulfill in the context. As a case study, we use these features to harvest structured subject knowledge of geometry from textbooks. We conclude that the discourse and text layout features provide information that is complementary to lexical semantic information. Finally, we show that the harvested structured knowledge can be used to improve an existing solver for geometry problems, making it more accurate as well as more explainable.
Anthology ID:
J19-4002
Volume:
Computational Linguistics, Volume 45, Issue 4 - December 2019
Month:
December
Year:
2019
Address:
Cambridge, MA
Venue:
CL
SIG:
Publisher:
MIT Press
Note:
Pages:
627–665
Language:
URL:
https://aclanthology.org/J19-4002
DOI:
10.1162/coli_a_00360
Bibkey:
Cite (ACL):
Mrinmaya Sachan, Avinava Dubey, Eduard H. Hovy, Tom M. Mitchell, Dan Roth, and Eric P. Xing. 2019. Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks. Computational Linguistics, 45(4):627–665.
Cite (Informal):
Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks (Sachan et al., CL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/J19-4002.pdf