Staging instance, all changes can be removed at any time

Skip to content

Numerous git-checkout loadings are failing in production while they should not

After the deployment to production of the new nixguix lister and related loaders, numerous loadings with the git-checkout visit_type have failed.

The reported errors are related to NAR checksum mismatch between the one computed by the git-checkout loader and the one provided by guix or nix in their sources.json manifest.

However there is something odd as if we try to load these repositories locally or in the docker environment, the loadings succeed without hash mismatches.

For instance for the loading of https://github.com/tdf/libcmis at tag v0.6.0, the following error is reported in production:

Checksum mismatched on <https://github.com/tdf/libcmis>: {'sha256': '512b6149a8e249ffedea1977f3239ce55247b0e66e97f7566f37b4df9c4a26f1'} != {'sha256': '136038b8951acaa30cc5589fcde01e60a2cbf85895daf4a134d7577b964b5d9e'}

While executing the loading locally is fine:

$ swh -l DEBUG loader run git-checkout https://github.com/tdf/libcmis ref=v0.6.0 checksum_layout=nar checksums='{"sha256": "136038b8951acaa30cc5589fcde01e60a2cbf85895daf4a134d7577b964b5d9e"}'
DEBUG:swh.loader.cli:ctx: <click.core.Context object at 0x7fd3f105d2d0>
DEBUG:swh.loader.cli:config_file: None
DEBUG:swh.loader.cli:config: {}
WARNING:swh.loader.cli:No storage configuration detected, using an in-memory storage instead.
DEBUG:swh.loader.cli:kw: {'ref': 'v0.6.0', 'checksum_layout': 'nar', 'checksums': {'sha256': '136038b8951acaa30cc5589fcde01e60a2cbf85895daf4a134d7577b964b5d9e'}}
DEBUG:swh.loader.cli:registry: {'task_modules': [], 'loader': <class 'swh.loader.git.directory.GitCheckoutLoader'>}
DEBUG:swh.loader.cli:loader class: <class 'swh.loader.git.directory.GitCheckoutLoader'>
DEBUG:swh.loader.git.directory.GitCheckoutLoader:Loader checksums computation: nar
INFO:swh.loader.git.directory.GitCheckoutLoader:Load origin 'https://github.com/tdf/libcmis' with type 'git-checkout'
DEBUG:swh.loader.git.directory.GitCheckoutLoader:lister_not provided, skipping extrinsic origin metadata
DEBUG:swh.loader.git.directory.GitCheckoutLoader:Artifact <git-checkout> with path /tmp/tmpl4k7e1k8/libcmis
DEBUG:swh.loader.git.directory.GitCheckoutLoader:Artifact <git-checkout> to check nar hashes: /tmp/tmpl4k7e1k8/libcmis
DEBUG:swh.loader.git.directory.GitCheckoutLoader:Number of skipped contents: 0
DEBUG:swh.loader.git.directory.GitCheckoutLoader:Number of contents: 333
DEBUG:swh.loader.git.directory.GitCheckoutLoader:Number of directories: 21
DEBUG:swh.loader.git.directory.GitCheckoutLoader:cleanup
{'status': 'eventful'} for origin 'https://github.com/tdf/libcmis'

I attached below the list of failing origin URLs extracted from sentry with that script for both staging and production environments:

To be noted, the origins that failed on staging also failed on production so the issue seems deterministic.