Staging instance, all changes can be removed at any time

Skip to content

luigi: Rewrite compression pipeline as small Luigi tasks

vlorentz requested to merge vlorentz/swh-graph:compress-steps into master

with one task for each step, instead of a single task running all steps.

This allows recovery on error, (negligeable) parallelization, and better progress reporting

Merge request reports