nixguix: fails to use previous visit snapshot
The next visits of nixguix loaders fails with the following. loader-nixguix version 0.0.90 [4]
Apr 22 07:36:21 worker0 python3[26357]: File "/usr/lib/python3/dist-packages/swh/loader/package/nixguix/tasks.py", line 14, in load_nixguix
Apr 22 07:36:21 worker0 python3[26357]: return NixGuixLoader(url).load()
Apr 22 07:36:21 worker0 python3[26357]: File "/usr/lib/python3/dist-packages/swh/loader/package/loader.py", line 328, in load
Apr 22 07:36:21 worker0 python3[26357]: revision_id = self.resolve_revision_from(known_artifacts, p_info["raw"])
Apr 22 07:36:21 worker0 python3[26357]: File "/usr/lib/python3/dist-packages/swh/loader/package/nixguix/loader.py", line 136, in resolve_revision_from
Apr 22 07:36:21 worker0 python3[26357]: known_integrity = known_artifact["extrinsic"]["raw"]["integrity"]
Apr 22 07:36:21 worker0 python3[26357]: KeyError: 'integrity'
Apr 22 07:36:21 worker0 python3[26362]: [2020-04-22 07:36:21,942: ERROR/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[7d942ed8-d811-4ffe-8971-d1d8fee0a3f7] raised unexpected: KeyError('integrity')
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 382, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/lib/python3/dist-packages/swh/scheduler/task.py", line 53, in __call__
result = super().__call__(*args, **kwargs)
File "/usr/lib/python3/dist-packages/celery/app/trace.py", line 641, in __protected_call__
return self.run(*args, **kwargs)
File "/usr/lib/python3/dist-packages/sentry_sdk/integrations/celery.py", line 161, in _inner
reraise(*exc_info)
File "/usr/lib/python3/dist-packages/sentry_sdk/_compat.py", line 57, in reraise
raise value
File "/usr/lib/python3/dist-packages/sentry_sdk/integrations/celery.py", line 156, in _inner
return f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/swh/loader/package/nixguix/tasks.py", line 14, in load_nixguix
return NixGuixLoader(url).load()
File "/usr/lib/python3/dist-packages/swh/loader/package/loader.py", line 328, in load
revision_id = self.resolve_revision_from(known_artifacts, p_info["raw"])
File "/usr/lib/python3/dist-packages/swh/loader/package/nixguix/loader.py", line 136, in resolve_revision_from
known_integrity = known_artifact["extrinsic"]["raw"]["integrity"]
KeyError: 'integrity'
Checking the sources-unstable.json used by the loader, nothing is amiss there
- [1].
Checking the snapshot branches metadata field, those are incomplete. They are missing the integrity field indeed [2] [3]
- [1]
$ wget https://nix-community.github.io/nixpkgs-swh/sources-unstable.json
$ ipython
In [2]: import json
In [3]: with open('sources-unstable.json') as f:
...: data = json.load(f)
...:
In [4]: missing_integrity=[]
...: for s in data:
...: if isinstance(s, dict) and 'integrity' not in s:
...: missing_integrity.append(s)
...:
In [5]: missing_integrity
Out[5]: []-
[4]
$ dpkg -l python3-swh.loader.core
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=======================-=====================-============-=================================
ii python3-swh.loader.core 0.0.90-1~swh1~bpo10+1 all Software Heritage Loader CoreMigrated from T2371 (view on Phabricator)
Edited by Phabricator Migration user