A Parallel Corpus of Python Functions and Documentation Strings for Automated Code Documentation and Code Generation

Antonio Valerio Miceli Barone, Rico Sennrich

Abstract
Automated documentation of programming source code and automated code generation from natural language are challenging tasks of both practical and scientific interest. Progress in these areas has been limited by the low availability of parallel corpora of code and natural language descriptions, which tend to be small and constrained to specific domains. In this work we introduce a large and diverse parallel corpus of a hundred thousands Python functions with their documentation strings (“docstrings”) generated by scraping open source repositories on GitHub. We describe baseline results for the code documentation and code generation tasks obtained by neural machine translation. We also experiment with data augmentation techniques to further increase the amount of training data. We release our datasets and processing scripts in order to stimulate research in these areas.
Anthology ID:
I17-2053
Volume:
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
November
Year:
2017
Address:
Taipei, Taiwan
Editors:
Greg Kondrak, Taro Watanabe
Venue:
IJCNLP
SIG:
Publisher:
Asian Federation of Natural Language Processing
Note:
Pages:
314–319
Language:
URL:
https://aclanthology.org/I17-2053
DOI:
Bibkey:
Cite (ACL):
Antonio Valerio Miceli Barone and Rico Sennrich. 2017. A Parallel Corpus of Python Functions and Documentation Strings for Automated Code Documentation and Code Generation. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 314–319, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Cite (Informal):
A Parallel Corpus of Python Functions and Documentation Strings for Automated Code Documentation and Code Generation (Miceli Barone & Sennrich, IJCNLP 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/improve-issue-templates/I17-2053.pdf
Code
 Avmb/code-docstring-corpus +  additional community code
Data
Django, Hearthstone
PDF Search Code