daminglu123 at SemEval-2022 Task 2: Using BERT and LSTM to Do Text Classification

Daming Lu


Abstract
Multiword expressions (MWEs) or idiomaticity are common phenomenon in natural languages. Current pre-trained language models cannot effectively capture the meaning of these MWEs. The reason is that two normal words, after combining together, could have an abruptly different meaning than the compositionality of the meanings of each word, whereas pre-trained language models reply on words compositionality. We proposed an improved method of adding an LSTM layer to the BERT model in order to get better results on a text classification task (Subtask A). Our result is slightly better than the baseline. We also tried adding TextCNN to BERT and adding both LSTM and TextCNN to BERT. We find that adding only LSTM gives the best performance.
Anthology ID:
2022.semeval-1.22
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
SemEval
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
186–189
Language:
URL:
https://aclanthology.org/2022.semeval-1.22
DOI:
10.18653/v1/2022.semeval-1.22
Bibkey:
Cite (ACL):
Daming Lu. 2022. daminglu123 at SemEval-2022 Task 2: Using BERT and LSTM to Do Text Classification. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 186–189, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
daminglu123 at SemEval-2022 Task 2: Using BERT and LSTM to Do Text Classification (Lu, SemEval 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/2022.semeval-1.22.pdf
Video:
 https://preview.aclanthology.org/auto-file-uploads/2022.semeval-1.22.mp4