Abstract
Recent success of pre-trained language models (PLMs) has stimulated interest in their ability to understand and work with numbers. Yet, the numerical reasoning over measurements has not been formally studied despite their importance. In this study, we show that PLMs lack the capability required for reasoning over measurements. Furthermore, we find that a language model trained on a measurement-rich corpus shows better performance on understanding measurements. We propose a simple embedding strategy to better distinguish between numbers and units, which leads to a significant improvement in the probing tasks.- Anthology ID:
- 2022.findings-emnlp.128
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2022
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1782–1792
- Language:
- URL:
- https://aclanthology.org/2022.findings-emnlp.128
- DOI:
- Cite (ACL):
- Sungjin Park, Seungwoo Ryu, and Edward Choi. 2022. Do Language Models Understand Measurements?. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1782–1792, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Do Language Models Understand Measurements? (Park et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.findings-emnlp.128.pdf