Abstract
For the last 5 years, we have developed and maintained RSMTool – an open-source tool for evaluating NLP systems that automatically score written and spoken responses. RSMTool is designed to be cross-disciplinary, borrowing heavily from NLP, machine learning, and educational measurement. Its cross-disciplinary nature has required us to learn a user-centered development approach in terms of both design and implementation. We share some of these lessons in this paper.- Anthology ID:
- 2020.nlposs-1.20
- Volume:
- Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Eunjeong L. Park, Masato Hagiwara, Dmitrijs Milajevs, Nelson F. Liu, Geeticka Chauhan, Liling Tan
- Venue:
- NLPOSS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 141–146
- Language:
- URL:
- https://aclanthology.org/2020.nlposs-1.20
- DOI:
- 10.18653/v1/2020.nlposs-1.20
- Cite (ACL):
- Nitin Madnani and Anastassia Loukina. 2020. User-centered & Robust NLP OSS: Lessons Learned from Developing & Maintaining RSMTool. In Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS), pages 141–146, Online. Association for Computational Linguistics.
- Cite (Informal):
- User-centered & Robust NLP OSS: Lessons Learned from Developing & Maintaining RSMTool (Madnani & Loukina, NLPOSS 2020)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2020.nlposs-1.20.pdf