Audit Report Coverage Assessment using Sentence Classification
Sushodhan Vaishampayan, Nitin Ramrakhiyani, Sachin Pawar, Aditi Pawde, Manoj Apte, Girish Palshikar
Abstract
Audit reports are a window to the financial health of a company and hence gauging coverage of various audit aspects in them is important. In this paper, we aim at determining an audit report’s coverage through classification of its sentences into multiple domain specific classes. In a weakly supervised setting, we employ a rule-based approach to automatically create training data for a BERT-based multi-label classifier. We then devise an ensemble to combine both the rule based and classifier approaches. Further, we employ two novel ways to improve the ensemble’s generalization: (i) through an active learning based approach and, (ii) through a LLM based review. We demonstrate that our proposed approaches outperform several baselines. We show utility of the proposed approaches to measure audit coverage on a large dataset of 2.8K audit reports.- Anthology ID:
- 2023.finnlp-2.4
- Volume:
- Proceedings of the Sixth Workshop on Financial Technology and Natural Language Processing
- Month:
- November
- Year:
- 2023
- Address:
- Bali, Indonesia
- Editors:
- Chung-Chi Chen, Hen-Hsen Huang, Hiroya Takamura, Hsin-Hsi Chen, Hiroki Sakaji, Kiyoshi Izumi
- Venues:
- FinNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 31–41
- Language:
- URL:
- https://aclanthology.org/2023.finnlp-2.4
- DOI:
- 10.18653/v1/2023.finnlp-2.4
- Cite (ACL):
- Sushodhan Vaishampayan, Nitin Ramrakhiyani, Sachin Pawar, Aditi Pawde, Manoj Apte, and Girish Palshikar. 2023. Audit Report Coverage Assessment using Sentence Classification. In Proceedings of the Sixth Workshop on Financial Technology and Natural Language Processing, pages 31–41, Bali, Indonesia. Association for Computational Linguistics.
- Cite (Informal):
- Audit Report Coverage Assessment using Sentence Classification (Vaishampayan et al., FinNLP-WS 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.finnlp-2.4.pdf