loader.BaseDirectoryLoader: Allow build_snapshot override

The current and default snapshot built is fine for the TarballDirectoryLoader or SvnDirectoryLoader. But we may have to improve the snapshot for directory coming from other VCS (where they can be associated to tag, changeset or release).

That won't be necessary for svn directory loader as all svn revisions are numerical. [1] But for git or hg, we can find commits, tag (or branch). [2]

This also adds another commit to add the fetch nar extids from checksums utility test fn to deduplicate code in the loaders' tests.

Refs. swh/meta#4979 (closed)

[1]

In [1]: import json

In [2]: filepath = '/home/tony/downloads/swh-guix/20230426-0946-sources.json'

In [3]: with open(filepath, 'r') as f: guix_data = json.loads(f.read())['sources']

In [32]: set(o['svn_revision'] for o in origins_svn)
Out[32]:
{170,
 353,
 399,
 401,
 505,
 603,
 1550,
 2341,
 2411,
 2965,
 18646,
 38938,
 39055,
 39057,
 39162,
 39263,
 39266,
 59745,
 62885,
 227977,
 261907,
 1807396}

sources.json is generated locally from the guix version with the upstream patch. Using the upstream sources.json should be fine now (as it merged upstream too). 20230426-0946-sources.json

[2]

In [30]: origins_git = [origin for origin in guix_data if origin['type'] == 'git' ]

In [31]: list(set(o['git_ref'] for o in origins_git))[0:10]
Out[31]:
['5.0.2',
 '6f02e58312654ddf19d686f234dcf699fc4cd0f5',
 '62112a7259e24bd6c08885629a185afe512b7d3d',
 'ea06abcca2a73a9779bcfb09081e56665f94e22a',
 '4.2.8',
 'v0.0.12',
 '0.6.2',
 'fba3567be95ed6e782d98a4c1477d3a74b8ad124',
 'e7efc48',
 '3ccfb407af410ba7e54ea0da11ae1e40b554a6f4']

In [37]: origins_hg = [origin for origin in guix_data if origin['type'] == 'hg' ]

In [36]: set(o['hg_changeset'] for o in origins_hg)
Out[36]:
{'0.2.6',
 '1.1',
 '1.10.1',
 '10.3.2',
 '178f55cf631a',
 '1bd98abda2ae4ef8f36761a167903e55c6bda7bb',
 '1e02c7614713bd0866c46f0c679a058f8c6d627e',
 '2e37299b13d8',
 '3.0.9',
 '4f8e60c03598f49aff6f5beeab40f2b995377e9f',
 '5048480a61243e6f1b02884012c8f25cdbee6d97',
 '5887634ea695',
 '67f1d1f22625',
 '765735bb590b',
 '89ad444c000b',
 '8u202-ga',
 '95b46b8428d5',
 'RELEASE_8_1_0',
 'Release_5_35',
 'b53eb6862bff',
 'd8df3bebc4b529388b62b9cb4df152f13910fbe3',
 'fd6845384b86',
 'jdk-10+46',
 'jdk-9+181',
 'v0.6.2',
 'v1.0.11',
 'v1.0.7',
 'v1.1',
 'v1.3',
 'v7.2.0-release'}