Staging instance, all changes can be removed at any time

Skip to content

sphinx/conf.py: Speedup documentation development build significantly

Antoine Lambert requested to merge anlambert/swh-docs:speedup-doc-build into master

The pydata sphinx theme we are using generates a custom sidebar for each documentation page but it is known that this step takes a long time if the whole documentation TOC tree has a large size (see pydata-sphinx-theme documentation and the flamegraph below I generated to profile the sphinx-build execution ).

profile.svg

So apply workaround suggested by the pydata sphinx theme documentation: remove from the TOC tree the API documentation pages generated by apidoc, except the root package indices, in order to significantly reduce the TOC tree size and thus the build time of the documentation.

Also rework the sphinx hook to include API reference of a package in its documentation index: prefer to modify source of documentation index instead of hacking the TOC tree and include links to submodules API.

Before:

(swh) ✔ ~/swh/swh-environment/swh-docs/docs [master|⚑ 9] 
$ time make
...
real    13m27,279s
user    14m23,286s
sys     0m9,899s

After:

(swh) ✔ ~/swh/swh-environment/swh-docs/docs [speedup-doc-build L|⚑ 9] 
$ time make
...
real    4m32,027s
user    5m29,672s
sys     0m9,270s

This performance optimization is only applied to the development builds of the full SWH documentation, either when calling make in the root directory of swh-docs or on Jenkins when pushing a merge request. For the latter, we gain more than 10 minutes for building the full SWH documentation, see screenshot below.

image

Edited by Antoine Lambert

Merge request reports

Loading