CodeWiki: Evaluating AI’s Ability to Generate Holistic Documentation for Large-Scale Codebases

Anh Nguyen Hoang, Minh Le-Anh, Bach Le, Nghi D. Q. Bui


Abstract
Comprehensive software documentation is crucial yet costly to produce. Despite recent advances in large language models (LLMs), generating holistic, architecture-aware documentation at the repository level remains challenging due to complex and evolving codebases that exceed LLM context limits. Existing automated methods struggle to capture rich semantic dependencies and architectural structure. We present CodeWiki, a unified framework for automated repository-level documentation across seven mainstream programming languages. CodeWiki combines top-down hierarchical decomposition with a divide-and-conquer agent system to preserve architectural context and scale documentation generation, and a bottom-up synthesis that integrates textual descriptions with visual artifacts such as architecture and data-flow diagrams. We also introduce CodeWikiBench, a benchmark with hierarchical rubrics and LLM-based evaluation protocols. Experiments show that CodeWiki achieves a 68.79% quality score with proprietary models, outperforming the closed-source DeepWiki baseline by 4.73%, with especially strong gains on scripting languages. CodeWiki is released as open source to support future research.
Anthology ID:
2026.findings-acl.288
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5812–5827
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.288/
DOI:
Bibkey:
Cite (ACL):
Anh Nguyen Hoang, Minh Le-Anh, Bach Le, and Nghi D. Q. Bui. 2026. CodeWiki: Evaluating AI’s Ability to Generate Holistic Documentation for Large-Scale Codebases. In Findings of the Association for Computational Linguistics: ACL 2026, pages 5812–5827, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
CodeWiki: Evaluating AI’s Ability to Generate Holistic Documentation for Large-Scale Codebases (Hoang et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.288.pdf
Checklist:
 2026.findings-acl.288.checklist.pdf