sphinx/conf.py: Speedup documentation development build significantly
The pydata sphinx theme we are using generates a custom sidebar for each
documentation page but it is known that this step takes a long time if
the whole documentation TOC tree has a large size (see pydata-sphinx-theme documentation
and the flamegraph below I generated to profile the sphinx-build
execution ).
So apply workaround suggested by the pydata sphinx theme documentation: remove from the TOC tree the API documentation pages generated by apidoc, except the root package indices, in order to significantly reduce the TOC tree size and thus the build time of the documentation.
Also rework the sphinx hook to include API reference of a package in its documentation index: prefer to modify source of documentation index instead of hacking the TOC tree and include links to submodules API.
Before:
(swh) ✔ ~/swh/swh-environment/swh-docs/docs [master|⚑ 9]
$ time make
...
real 13m27,279s
user 14m23,286s
sys 0m9,899s
After:
(swh) ✔ ~/swh/swh-environment/swh-docs/docs [speedup-doc-build L|⚑ 9]
$ time make
...
real 4m32,027s
user 5m29,672s
sys 0m9,270s
This performance optimization is only applied to the development builds of the full SWH documentation,
either when calling make
in the root directory of swh-docs
or on Jenkins when pushing a merge request.
For the latter, we gain more than 10 minutes for building the full SWH documentation, see screenshot below.