Abstract
This paper presents our winning solution for the Shared Task on Detecting Signs of Depression from Social Media Text at LT-EDI-ACL2022. The task was to create a system that, given social media posts in English, should detect the level of depression as ‘not depressed’, ‘moderately depressed’ or ‘severely depressed’. We based our solution on transformer-based language models. We fine-tuned selected models: BERT, RoBERTa, XLNet, of which the best results were obtained for RoBERTa. Then, using the prepared corpus, we trained our own language model called DepRoBERTa (RoBERTa for Depression Detection). Fine-tuning of this model improved the results. The third solution was to use the ensemble averaging, which turned out to be the best solution. It achieved a macro-averaged F1-score of 0.583. The source code of prepared solution is available at https://github.com/rafalposwiata/depression-detection-lt-edi-2022.- Anthology ID:
- 2022.ltedi-1.40
- Volume:
- Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Venue:
- LTEDI
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 276–282
- Language:
- URL:
- https://aclanthology.org/2022.ltedi-1.40
- DOI:
- 10.18653/v1/2022.ltedi-1.40
- Cite (ACL):
- Rafał Poświata and Michał Perełkiewicz. 2022. OPI@LT-EDI-ACL2022: Detecting Signs of Depression from Social Media Text using RoBERTa Pre-trained Language Models. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, pages 276–282, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- OPI@LT-EDI-ACL2022: Detecting Signs of Depression from Social Media Text using RoBERTa Pre-trained Language Models (Poświata & Perełkiewicz, LTEDI 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.ltedi-1.40.pdf
- Code
- rafalposwiata/depression-detection-lt-edi-2022