Staging instance, all changes can be removed at any time

Skip to content

loader: Implement resolve_ext_ref callback of PackInflater

It exist cases where a git server can send a pack file containing external references to objects stored in the local repository. In that case, data for such objects are not included in the pack file and must be resolved from the local git object store.

Such a pack file is typically generated when a git client asks for a thin pack to the server but can also be generated when a client is not requesting a thin pack.

It has been observed that GitHub can send such pack files to the loader for origins already visited. The loader does not ask for thin pack files so it might be an implementation issue of git-upload-pack from their side.

This is problematic as dulwich raises a KeyError exception when it finds a pack file has external references not resolved in it and thus the git loading fails.

In order to workaround that issue, implement the resolve_ext_ref callback that can be passed as parameter to the PackInflater class. When dulwich encounters an external reference in the pack file, it is calling that callback to resolve object data. As an external object has already been stored into the archive, we can reconstruct its git manifest from the archive content and thus resolve the external reference in the pack file.

Fixes #4745 (closed)

As a side note, theses changes now allow the loader to ask for thin packs but I am not sure if there is any benefits for us to do so.

Merge request reports

Loading