Discovering Better Model Architectures for Medical Query Understanding

Wei Zhu, Yuan Ni, Xiaoling Wang, Guotong Xie


Abstract
In developing an online question-answering system for the medical domains, natural language inference (NLI) models play a central role in question matching and intention detection. However, which models are best for our datasets? Manually selecting or tuning a model is time-consuming. Thus we experiment with automatically optimizing the model architectures on the task at hand via neural architecture search (NAS). First, we formulate a novel architecture search space based on the previous NAS literature, supporting cross-sentence attention (cross-attn) modeling. Second, we propose to modify the ENAS method to accelerate and stabilize the search results. We conduct extensive experiments on our two medical NLI tasks. Results show that our system can easily outperform the classical baseline models. We compare different NAS methods and demonstrate our approach provides the best results.
Anthology ID:
2021.naacl-industry.29
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers
Month:
June
Year:
2021
Address:
Online
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
230–237
Language:
URL:
https://aclanthology.org/2021.naacl-industry.29
DOI:
10.18653/v1/2021.naacl-industry.29
Bibkey:
Cite (ACL):
Wei Zhu, Yuan Ni, Xiaoling Wang, and Guotong Xie. 2021. Discovering Better Model Architectures for Medical Query Understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers, pages 230–237, Online. Association for Computational Linguistics.
Cite (Informal):
Discovering Better Model Architectures for Medical Query Understanding (Zhu et al., NAACL 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.naacl-industry.29.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2021.naacl-industry.29.mp4