Abstract
The pre-trained BERT model achieves a remarkable state of the art across a wide range of tasks in natural language processing. For solving the gender bias in gendered pronoun resolution task, I propose a novel neural network model based on the pre-trained BERT. This model is a type of mention score classifier and uses an attention mechanism with no parameters to compute the contextual representation of entity span, and a vector to represent the triple-wise semantic similarity among the pronoun and the entities. In stage 1 of the gendered pronoun resolution task, a variant of this model, trained in the fine-tuning approach, reduced the multi-class logarithmic loss to 0.3033 in the 5-fold cross-validation of training set and 0.2795 in testing set. Besides, this variant won the 2nd place with a score at 0.17289 in stage 2 of the task. The code in this paper is available at: https://github.com/ziliwang/MSnet-for-Gendered-Pronoun-Resolution- Anthology ID:
- W19-3813
- Volume:
- Proceedings of the First Workshop on Gender Bias in Natural Language Processing
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Venue:
- GeBNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 89–95
- Language:
- URL:
- https://aclanthology.org/W19-3813
- DOI:
- 10.18653/v1/W19-3813
- Cite (ACL):
- Zili Wang. 2019. MSnet: A BERT-based Network for Gendered Pronoun Resolution. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 89–95, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- MSnet: A BERT-based Network for Gendered Pronoun Resolution (Wang, GeBNLP 2019)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/W19-3813.pdf
- Code
- ziliwang/MSnet-for-Gendered-Pronoun-Resolution
- Data
- GAP Coreference Dataset