Work around issue on frozen set generation
This is the sole solution i've managed to make build successfully images.
That falsifies the frozen set definition... But I'd argue for it as a temporary solution which unstucks the build docker image pipeline development...
I've tried other implementations to no avail (pushed in another branch in my fork)
- docker (during the image build that runs the script) [1]
- venv (with app_manager.py) [2]
frozen set generation and build commands
$ docker build -t app-manager scripts/ Sending build context to Docker daemon 19.97kB Step 1/7 : FROM python:3.10-bullseye ---> 109421b4b8cd Step 2/7 : RUN apt-get -y update && apt-get -y upgrade && apt-get install -y libcmph-dev librdkafka-dev libsvn-dev libapr1-dev libaprutil1-dev libpython3-dev libpq-dev python3-click python3-venv python3-yaml python3-dulwich && apt clean ---> Using cache ---> b3fb8e0051cf Step 3/7 : RUN mkdir -p /opt/scripts/ ---> Using cache ---> aa645b2ebd30 Step 4/7 : COPY app_manager.py /opt/scripts/ ---> f7e9348f4437 Step 5/7 : RUN chmod +x /opt/scripts/app_manager.py ---> Running in 590749456dab Removing intermediate container 590749456dab ---> 99f3875e7fe0 Step 6/7 : ENV SWH_APPS_DIR=/src/apps ---> Running in ce8ba9d4b9a0 Removing intermediate container ce8ba9d4b9a0 ---> b0344df6431c Step 7/7 : ENTRYPOINT [ "/opt/scripts/app_manager.py" ] ---> Running in e7267e1b1233 Removing intermediate container e7267e1b1233 ---> 8d0c5050739f Successfully built 8d0c5050739f Successfully tagged app-manager:latest$ git status
master...upstream/master
M scripts/app_manager.py
$ docker run -v $PWD:/src --user 1000 -it app-manager generate-frozen-requirements swh-loader-git Requirement already satisfied: pip in /tmp/swh-loader-gitu15uqnzc/lib/python3.9/site-packages (20.3.4) Collecting pip Downloading pip-22.3.1-py3-none-any.whl (2.1 MB) |████████████████████████████████| 2.1 MB 16.2 MB/s Requirement already satisfied: setuptools in /tmp/swh-loader-gitu15uqnzc/lib/python3.9/site-packages (44.1.1) Collecting setuptools Downloading setuptools-66.1.1-py3-none-any.whl (1.3 MB) |████████████████████████████████| 1.3 MB 110.7 MB/s Collecting wheel Downloading wheel-0.38.4-py3-none-any.whl (36 kB) Installing collected packages: wheel, setuptools, pip Attempting uninstall: setuptools Found existing installation: setuptools 44.1.1 Uninstalling setuptools-44.1.1: Successfully uninstalled setuptools-44.1.1 Attempting uninstall: pip Found existing installation: pip 20.3.4 Uninstalling pip-20.3.4: Successfully uninstalled pip-20.3.4 Successfully installed pip-22.3.1 setuptools-66.1.1 wheel-0.38.4 Collecting swh-loader-git Downloading swh.loader.git-2.1.0-py3-none-any.whl (73 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.8/73.8 kB 20.5 MB/s eta 0:00:00 Collecting swh-loader-metadata Downloading swh.loader.metadata-0.0.3-py3-none-any.whl (34 kB) Collecting swh.scheduler>=0.0.39 Downloading swh.scheduler-1.5.0-py3-none-any.whl (166 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 kB 89.6 MB/s eta 0:00:00 Collecting retrying Downloading retrying-1.3.4-py3-none-any.whl (11 kB) Collecting dulwich>=0.20.43 Downloading dulwich-0.21.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (505 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 505.7/505.7 kB 183.1 MB/s eta 0:00:00 Collecting swh.storage>=0.22.0 Downloading swh.storage-1.7.3-py3-none-any.whl (449 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 449.7/449.7 kB 368.5 MB/s eta 0:00:00 Collecting click Downloading click-8.1.3-py3-none-any.whl (96 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 358.4 MB/s eta 0:00:00 Collecting swh.model>=4.3.0 Downloading swh.model-6.6.1-py3-none-any.whl (136 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 136.2/136.2 kB 257.2 MB/s eta 0:00:00 Collecting swh.loader.core>=5.0.0 Downloading swh.loader.core-5.1.0-py3-none-any.whl (10.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.0/10.0 MB 109.9 MB/s eta 0:00:00 Collecting swh.core>=0.0.7 Downloading swh.core-2.20.0-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 130.0 MB/s eta 0:00:00 Collecting swh.lister>=2.9.0 Downloading swh.lister-4.3.3-py3-none-any.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 141.9 MB/s eta 0:00:00 Collecting urllib3>=1.25 Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 kB 253.4 MB/s eta 0:00:00 Collecting pyyaml Downloading PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 661.8/661.8 kB 182.6 MB/s eta 0:00:00 Collecting python-magic Downloading python_magic-0.4.27-py2.py3-none-any.whl (13 kB) Collecting deprecated Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB) Collecting sentry-sdk Downloading sentry_sdk-1.14.0-py2.py3-none-any.whl (178 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.9/178.9 kB 390.3 MB/s eta 0:00:00 Collecting aiohttp Downloading aiohttp-3.8.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 166.5 MB/s eta 0:00:00 Collecting requests Downloading requests-2.28.2-py3-none-any.whl (62 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 283.0 MB/s eta 0:00:00 Collecting flask Downloading Flask-2.2.2-py3-none-any.whl (101 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.5/101.5 kB 365.0 MB/s eta 0:00:00 Collecting blinker Downloading blinker-1.5-py2.py3-none-any.whl (12 kB) Collecting msgpack>=1.0.0 Downloading msgpack-1.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (322 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 322.4/322.4 kB 367.2 MB/s eta 0:00:00 Collecting iso8601 Downloading iso8601-1.1.0-py3-none-any.whl (9.9 kB) Collecting aiohttp-utils>=3.1.1 Downloading aiohttp_utils-3.1.1-py2.py3-none-any.whl (10 kB) Collecting launchpadlib Downloading launchpadlib-1.11.0-py2.py3-none-any.whl (217 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 217.7/217.7 kB 256.6 MB/s eta 0:00:00 Requirement already satisfied: setuptools in /tmp/swh-loader-gitu15uqnzc/lib/python3.9/site-packages (from swh.lister>=2.9.0->swh-loader-metadata->-r /src/apps/swh-loader-git/requirements.txt (line 2)) (66.1.1) Collecting beautifulsoup4 Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.2/128.2 kB 218.1 MB/s eta 0:00:00 Collecting lxml Downloading lxml-4.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (7.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 113.4 MB/s eta 0:00:00 Collecting testing.postgresql Downloading testing.postgresql-1.3.0-py2.py3-none-any.whl (8.9 kB) Collecting repomd Downloading repomd-0.2.1-py3-none-any.whl (3.9 kB) Collecting tenacity>=6.2 Downloading tenacity-8.1.0-py3-none-any.whl (23 kB) Collecting python-debian Downloading python_debian-0.1.49-py3-none-any.whl (132 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 212.4 MB/s eta 0:00:00 Collecting psycopg2 Downloading psycopg2-2.9.5.tar.gz (384 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 384.3/384.3 kB 229.2 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Collecting python-dateutil Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 388.7 MB/s eta 0:00:00 Collecting typing-extensions Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB) Collecting toml Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) Collecting pkginfo Downloading pkginfo-1.9.6-py3-none-any.whl (30 kB) Collecting swh.objstorage>=0.2.2 Downloading swh.objstorage-2.0.1-py3-none-any.whl (93 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.7/93.7 kB 375.8 MB/s eta 0:00:00 Collecting psutil Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 280.2/280.2 kB 391.3 MB/s eta 0:00:00 Collecting packaging Downloading packaging-23.0-py3-none-any.whl (42 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.7/42.7 kB 266.1 MB/s eta 0:00:00 Collecting hypothesis Downloading hypothesis-6.65.0-py3-none-any.whl (402 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 402.2/402.2 kB 304.2 MB/s eta 0:00:00 Collecting attrs-strict>=0.0.7 Downloading attrs_strict-1.0.0-py3-none-any.whl (14 kB) Collecting attrs!=21.1.0 Downloading attrs-22.2.0-py3-none-any.whl (60 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.0/60.0 kB 223.5 MB/s eta 0:00:00 Collecting celery!=5.0.3,>=4.3 Downloading celery-5.2.7-py3-none-any.whl (405 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 405.6/405.6 kB 270.5 MB/s eta 0:00:00 Collecting humanize Downloading humanize-4.4.0-py3-none-any.whl (106 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 kB 281.0 MB/s eta 0:00:00 Collecting importlib-metadata<5.0 Downloading importlib_metadata-4.13.0-py3-none-any.whl (23 kB) Collecting pika>=1.1.0 Downloading pika-1.3.1-py3-none-any.whl (155 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 338.0 MB/s eta 0:00:00 Collecting cassandra-driver!=3.21.0,>=3.19.0 Downloading cassandra_driver-3.25.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (3.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 119.0 MB/s eta 0:00:00 Collecting redis Downloading redis-4.4.2-py3-none-any.whl (237 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 237.8/237.8 kB 295.8 MB/s eta 0:00:00 Collecting swh.counters>=v0.8.0 Downloading swh.counters-0.9.1-py3-none-any.whl (37 kB) Collecting mypy-extensions Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB) Collecting six>=1.7.0 Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting gunicorn Downloading gunicorn-20.1.0-py3-none-any.whl (79 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.5/79.5 kB 375.1 MB/s eta 0:00:00 Collecting python-mimeparse Downloading python_mimeparse-1.6.0-py2.py3-none-any.whl (6.1 kB) Collecting async-timeout<5.0,>=4.0.0a3 Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB) Collecting yarl<2.0,>=1.0 Downloading yarl-1.8.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 264.6/264.6 kB 368.1 MB/s eta 0:00:00 Collecting multidict<7.0,>=4.5 Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB 370.5 MB/s eta 0:00:00 Collecting frozenlist>=1.1.1 Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.8/158.8 kB 368.4 MB/s eta 0:00:00 Collecting charset-normalizer<3.0,>=2.0 Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB) Collecting aiosignal>=1.1.2 Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Collecting geomet<0.3,>=0.1 Downloading geomet-0.2.1.post1-py3-none-any.whl (18 kB) Collecting billiard<4.0,>=3.6.4.0 Downloading billiard-3.6.4.0-py3-none-any.whl (89 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.5/89.5 kB 362.2 MB/s eta 0:00:00 Collecting vine<6.0,>=5.0.0 Downloading vine-5.0.0-py2.py3-none-any.whl (9.4 kB) Collecting click-didyoumean>=0.0.3 Downloading click_didyoumean-0.3.0-py3-none-any.whl (2.7 kB) Collecting click-plugins>=1.1.1 Downloading click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB) Collecting kombu<6.0,>=5.2.3 Downloading kombu-5.2.4-py3-none-any.whl (189 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 189.2/189.2 kB 392.7 MB/s eta 0:00:00 Collecting click-repl>=0.2.0 Downloading click_repl-0.2.0-py3-none-any.whl (5.2 kB) Collecting pytz>=2021.3 Downloading pytz-2022.7.1-py2.py3-none-any.whl (499 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 499.4/499.4 kB 301.9 MB/s eta 0:00:00 Collecting zipp>=0.5 Downloading zipp-3.11.0-py3-none-any.whl (6.6 kB) Collecting pytest-postgresql<4.0.0,>=3 Downloading pytest_postgresql-3.1.3-py3-none-any.whl (41 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.6/41.6 kB 247.6 MB/s eta 0:00:00 Collecting swh.journal Downloading swh.journal-1.2.1-py3-none-any.whl (40 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 kB 235.1 MB/s eta 0:00:00 Collecting swh.perfecthash>=0.1.2 Downloading swh.perfecthash-0.1.2.tar.gz (27 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Collecting soupsieve>1.2 Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB) Collecting wrapt<2,>=1.10 Downloading wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.8/77.8 kB 261.6 MB/s eta 0:00:00 Collecting Werkzeug>=2.2.2 Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.7/232.7 kB 403.2 MB/s eta 0:00:00 Collecting itsdangerous>=2.0 Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB) Collecting Jinja2>=3.0 Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 252.1 MB/s eta 0:00:00 Collecting sortedcontainers<3.0.0,>=2.1.0 Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB) Collecting exceptiongroup>=1.0.0 Downloading exceptiongroup-1.1.0-py3-none-any.whl (14 kB) Collecting httplib2 Downloading httplib2-0.21.0-py3-none-any.whl (96 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.8/96.8 kB 358.6 MB/s eta 0:00:00 Collecting lazr.uri Downloading lazr.uri-1.0.6.tar.gz (18 kB) Preparing metadata (setup.py) ... done Collecting lazr.restfulclient>=0.14.2 Downloading lazr.restfulclient-0.14.5-py3-none-any.whl (70 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.6/70.6 kB 324.6 MB/s eta 0:00:00 Collecting chardet Downloading chardet-5.1.0-py3-none-any.whl (199 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.1/199.1 kB 261.3 MB/s eta 0:00:00 Collecting defusedxml Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB) Collecting certifi>=2017.4.17 Downloading certifi-2022.12.7-py3-none-any.whl (155 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 249.9 MB/s eta 0:00:00 Collecting idna<4,>=2.5 Downloading idna-3.4-py3-none-any.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 320.1 MB/s eta 0:00:00 Collecting pg8000>=1.10 Downloading pg8000-1.29.4-py3-none-any.whl (51 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.4/51.4 kB 250.0 MB/s eta 0:00:00 Collecting testing.common.database Downloading testing.common.database-2.0.3-py2.py3-none-any.whl (10 kB) Collecting prompt-toolkit Downloading prompt_toolkit-3.0.36-py3-none-any.whl (386 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 386.4/386.4 kB 286.8 MB/s eta 0:00:00 Collecting MarkupSafe>=2.0 Downloading MarkupSafe-2.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB) Collecting amqp<6.0.0,>=5.0.9 Downloading amqp-5.1.1-py3-none-any.whl (50 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.8/50.8 kB 234.7 MB/s eta 0:00:00 Collecting wadllib>=1.1.4 Downloading wadllib-1.3.6.tar.gz (62 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.2/62.2 kB 262.7 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Collecting oauthlib Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 327.6 MB/s eta 0:00:00 Collecting distro Downloading distro-1.8.0-py3-none-any.whl (20 kB) Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 258.7 MB/s eta 0:00:00 Collecting scramp>=1.4.3 Downloading scramp-1.4.4-py3-none-any.whl (13 kB) Collecting mirakuru>=2.3.0 Downloading mirakuru-2.4.2-py3-none-any.whl (29 kB) Collecting port-for Downloading port_for-0.6.3-py3-none-any.whl (21 kB) Collecting pytest>=6.2.0 Downloading pytest-7.2.1-py3-none-any.whl (317 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 kB 274.7 MB/s eta 0:00:00 Collecting cffi Downloading cffi-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.2/441.2 kB 228.3 MB/s eta 0:00:00 Collecting confluent-kafka Downloading confluent_kafka-2.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 116.4 MB/s eta 0:00:00 Collecting iniconfig Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB) Collecting pluggy<2.0,>=0.12 Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB) Collecting tomli>=1.0.0 Downloading tomli-2.0.1-py3-none-any.whl (12 kB) Collecting asn1crypto>=1.5.1 Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.0/105.0 kB 337.2 MB/s eta 0:00:00 Collecting pycparser Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 266.7 MB/s eta 0:00:00 Collecting wcwidth Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB) Building wheels for collected packages: psycopg2, swh.perfecthash, lazr.uri, wadllib Building wheel for psycopg2 (setup.py) ... done Created wheel for psycopg2: filename=psycopg2-2.9.5-cp39-cp39-linux_x86_64.whl size=456363 sha256=bd081f655ff0cd28a74633466d2f81ec9d7eb87b3a57d3ce648ca5e534151fa4 Stored in directory: /tmp/pip-ephem-wheel-cache-_8op_6pk/wheels/7e/ad/8b/26873413259428280c0894640ff3c475221f3b10282796e9d6 Building wheel for swh.perfecthash (pyproject.toml) ... done Created wheel for swh.perfecthash: filename=swh.perfecthash-0.1.2-cp39-cp39-linux_x86_64.whl size=44342 sha256=ba7966aed650d51975d106c30a3c05d9f2abe6102913b3b24c68c62a9825b669 Stored in directory: /tmp/pip-ephem-wheel-cache-_8op_6pk/wheels/07/49/c1/7c53ed0ea6208320a2d9ff6faf2fc1922304fccba86bdcc4e7 Building wheel for lazr.uri (setup.py) ... done Created wheel for lazr.uri: filename=lazr.uri-1.0.6-py3-none-any.whl size=20359 sha256=4dc47e7cf8c6297b71869dff150830b8952c4636b1334f66b53c76643579405c Stored in directory: /tmp/pip-ephem-wheel-cache-_8op_6pk/wheels/65/f1/1c/708e52900a3dfdc83b80cd112643516468d4671e40e0c906ad Building wheel for wadllib (setup.py) ... done Created wheel for wadllib: filename=wadllib-1.3.6-py3-none-any.whl size=58279 sha256=83c806fc6fce39f079bf99bd50f7c470b39eaafb4df5272928c87dbdb34fe114 Stored in directory: /tmp/pip-ephem-wheel-cache-_8op_6pk/wheels/15/8c/a2/fc4313c0655453812b92189cedd07b0bad20935f29306ae54d Successfully built psycopg2 swh.perfecthash lazr.uri wadllib Installing collected packages: wcwidth, testing.common.database, sortedcontainers, pytz, python-mimeparse, mypy-extensions, msgpack, confluent-kafka, billiard, asn1crypto, zipp, wrapt, vine, urllib3, typing-extensions, tomli, toml, tenacity, soupsieve, six, scramp, pyyaml, python-magic, pyparsing, pycparser, psycopg2, psutil, prompt-toolkit, port-for, pluggy, pkginfo, pika, packaging, oauthlib, multidict, MarkupSafe, lxml, lazr.uri, itsdangerous, iso8601, iniconfig, idna, humanize, gunicorn, frozenlist, exceptiongroup, distro, defusedxml, click, charset-normalizer, chardet, certifi, blinker, attrs, async-timeout, yarl, Werkzeug, wadllib, sentry-sdk, retrying, requests, repomd, redis, python-debian, python-dateutil, pytest, mirakuru, Jinja2, importlib-metadata, hypothesis, httplib2, geomet, dulwich, deprecated, click-repl, click-plugins, click-didyoumean, cffi, beautifulsoup4, attrs-strict, amqp, aiosignal, swh.perfecthash, swh.model, swh.core, pytest-postgresql, pg8000, lazr.restfulclient, kombu, flask, cassandra-driver, aiohttp, testing.postgresql, swh.journal, launchpadlib, celery, aiohttp-utils, swh.objstorage, swh.counters, swh.storage, swh.scheduler, swh.loader.core, swh.lister, swh-loader-metadata, swh-loader-git Successfully installed Jinja2-3.1.2 MarkupSafe-2.1.2 Werkzeug-2.2.2 aiohttp-3.8.3 aiohttp-utils-3.1.1 aiosignal-1.3.1 amqp-5.1.1 asn1crypto-1.5.1 async-timeout-4.0.2 attrs-22.2.0 attrs-strict-1.0.0 beautifulsoup4-4.11.1 billiard-3.6.4.0 blinker-1.5 cassandra-driver-3.25.0 celery-5.2.7 certifi-2022.12.7 cffi-1.15.1 chardet-5.1.0 charset-normalizer-2.1.1 click-8.1.3 click-didyoumean-0.3.0 click-plugins-1.1.1 click-repl-0.2.0 confluent-kafka-2.0.2 defusedxml-0.7.1 deprecated-1.2.13 distro-1.8.0 dulwich-0.21.2 exceptiongroup-1.1.0 flask-2.2.2 frozenlist-1.3.3 geomet-0.2.1.post1 gunicorn-20.1.0 httplib2-0.21.0 humanize-4.4.0 hypothesis-6.65.0 idna-3.4 importlib-metadata-4.13.0 iniconfig-2.0.0 iso8601-1.1.0 itsdangerous-2.1.2 kombu-5.2.4 launchpadlib-1.11.0 lazr.restfulclient-0.14.5 lazr.uri-1.0.6 lxml-4.9.2 mirakuru-2.4.2 msgpack-1.0.4 multidict-6.0.4 mypy-extensions-0.4.3 oauthlib-3.2.2 packaging-23.0 pg8000-1.29.4 pika-1.3.1 pkginfo-1.9.6 pluggy-1.0.0 port-for-0.6.3 prompt-toolkit-3.0.36 psutil-5.9.4 psycopg2-2.9.5 pycparser-2.21 pyparsing-3.0.9 pytest-7.2.1 pytest-postgresql-3.1.3 python-dateutil-2.8.2 python-debian-0.1.49 python-magic-0.4.27 python-mimeparse-1.6.0 pytz-2022.7.1 pyyaml-6.0 redis-4.4.2 repomd-0.2.1 requests-2.28.2 retrying-1.3.4 scramp-1.4.4 sentry-sdk-1.14.0 six-1.16.0 sortedcontainers-2.4.0 soupsieve-2.3.2.post1 swh-loader-git-2.1.0 swh-loader-metadata-0.0.3 swh.core-2.20.0 swh.counters-0.9.1 swh.journal-1.2.1 swh.lister-4.3.3 swh.loader.core-5.1.0 swh.model-6.6.1 swh.objstorage-2.0.1 swh.perfecthash-0.1.2 swh.scheduler-1.5.0 swh.storage-1.7.3 tenacity-8.1.0 testing.common.database-2.0.3 testing.postgresql-1.3.0 toml-0.10.2 tomli-2.0.1 typing-extensions-4.4.0 urllib3-1.26.14 vine-5.0.0 wadllib-1.3.6 wcwidth-0.2.6 wrapt-1.14.1 yarl-1.8.2 zipp-3.11.0
$ gdf | grep "pkg_resources" # prior to this workaround, that'd display the dreaded pkg_resources==0.0.0 line {+ """Work around issue with wheel generating wrong version for pkg_resources."""+} {+ return requirements_content.replace(b'pkg_resources==0.0.0\n', b'\n')+}
$ docker build --tag container-registry.softwareheritage.org/swh/infra/swh-apps/loader_git:20230124.1 apps/swh-loader-git Sending build context to Docker daemon 8.192kB Step 1/15 : FROM python:3.10-bullseye ---> 109421b4b8cd Step 2/15 : RUN apt-get -y update && apt-get -y upgrade && apt-get install -y libcmph-dev librdkafka-dev && apt clean && addgroup --gid 1000 swh && useradd --gid 1000 --uid 1000 -m -d /opt/swh swh && mkdir /etc/swh ---> Using cache ---> 5a6d2e1289bb Step 3/15 : USER swh ---> Using cache ---> 4c1fca783f9d Step 4/15 : WORKDIR /opt/swh ---> Using cache ---> bd167973351e Step 5/15 : COPY --chown=swh:swh requirements-frozen.txt /opt/swh ---> 1348926b5a97 Step 6/15 : COPY --chown=swh:swh entrypoint.sh /opt/swh ---> 9e415498f881 Step 7/15 : ENV PYTHONPATH=/opt/swh ---> Running in b70a8792d09c Removing intermediate container b70a8792d09c ---> 2f3ad33139c9 Step 8/15 : ENV PATH=/opt/swh/.local/bin:$PATH ---> Running in c51adb747254 Removing intermediate container c51adb747254 ---> f80645dd4b71 Step 9/15 : RUN chmod u+x /opt/swh/entrypoint.sh && /usr/local/bin/python -m pip install --upgrade pip && pip install --no-cache-dir -r requirements-frozen.txt && pip install gunicorn ---> Running in 7c09ab0c3117 Defaulting to user installation because normal site-packages is not writeable Requirement already satisfied: pip in /usr/local/lib/python3.10/site-packages (22.3.1) Defaulting to user installation because normal site-packages is not writeable Collecting aiohttp==3.8.3 Downloading aiohttp-3.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 56.1 MB/s eta 0:00:00 Collecting aiohttp-utils==3.1.1 Downloading aiohttp_utils-3.1.1-py2.py3-none-any.whl (10 kB) Collecting aiosignal==1.3.1 Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Collecting amqp==5.1.1 Downloading amqp-5.1.1-py3-none-any.whl (50 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.8/50.8 kB 382.1 MB/s eta 0:00:00 Collecting asn1crypto==1.5.1 Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.0/105.0 kB 439.9 MB/s eta 0:00:00 Collecting async-timeout==4.0.2 Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB) Collecting attrs==22.2.0 Downloading attrs-22.2.0-py3-none-any.whl (60 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.0/60.0 kB 257.1 MB/s eta 0:00:00 Collecting attrs-strict==1.0.0 Downloading attrs_strict-1.0.0-py3-none-any.whl (14 kB) Collecting beautifulsoup4==4.11.1 Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.2/128.2 kB 302.2 MB/s eta 0:00:00 Collecting billiard==3.6.4.0 Downloading billiard-3.6.4.0-py3-none-any.whl (89 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.5/89.5 kB 445.1 MB/s eta 0:00:00 Collecting blinker==1.5 Downloading blinker-1.5-py2.py3-none-any.whl (12 kB) Collecting cassandra-driver==3.25.0 Downloading cassandra_driver-3.25.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (3.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 113.8 MB/s eta 0:00:00 Collecting celery==5.2.7 Downloading celery-5.2.7-py3-none-any.whl (405 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 405.6/405.6 kB 202.8 MB/s eta 0:00:00 Collecting certifi==2022.12.7 Downloading certifi-2022.12.7-py3-none-any.whl (155 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 446.2 MB/s eta 0:00:00 Collecting cffi==1.15.1 Downloading cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.8/441.8 kB 224.6 MB/s eta 0:00:00 Collecting chardet==5.1.0 Downloading chardet-5.1.0-py3-none-any.whl (199 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.1/199.1 kB 453.5 MB/s eta 0:00:00 Collecting charset-normalizer==2.1.1 Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB) Collecting click==8.1.3 Downloading click-8.1.3-py3-none-any.whl (96 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 433.4 MB/s eta 0:00:00 Collecting click-didyoumean==0.3.0 Downloading click_didyoumean-0.3.0-py3-none-any.whl (2.7 kB) Collecting click-plugins==1.1.1 Downloading click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB) Collecting click-repl==0.2.0 Downloading click_repl-0.2.0-py3-none-any.whl (5.2 kB) Collecting confluent-kafka==2.0.2 Downloading confluent_kafka-2.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 62.5 MB/s eta 0:00:00 Collecting defusedxml==0.7.1 Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB) Collecting Deprecated==1.2.13 Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB) Collecting distro==1.8.0 Downloading distro-1.8.0-py3-none-any.whl (20 kB) Collecting dulwich==0.21.2 Downloading dulwich-0.21.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (506 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 506.2/506.2 kB 154.4 MB/s eta 0:00:00 Collecting exceptiongroup==1.1.0 Downloading exceptiongroup-1.1.0-py3-none-any.whl (14 kB) Collecting Flask==2.2.2 Downloading Flask-2.2.2-py3-none-any.whl (101 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.5/101.5 kB 445.3 MB/s eta 0:00:00 Collecting frozenlist==1.3.3 Downloading frozenlist-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.6/149.6 kB 331.1 MB/s eta 0:00:00 Collecting geomet==0.2.1.post1 Downloading geomet-0.2.1.post1-py3-none-any.whl (18 kB) Collecting gunicorn==20.1.0 Downloading gunicorn-20.1.0-py3-none-any.whl (79 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.5/79.5 kB 439.6 MB/s eta 0:00:00 Collecting httplib2==0.21.0 Downloading httplib2-0.21.0-py3-none-any.whl (96 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.8/96.8 kB 467.7 MB/s eta 0:00:00 Collecting humanize==4.4.0 Downloading humanize-4.4.0-py3-none-any.whl (106 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 kB 468.4 MB/s eta 0:00:00 Collecting hypothesis==6.65.0 Downloading hypothesis-6.65.0-py3-none-any.whl (402 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 402.2/402.2 kB 220.2 MB/s eta 0:00:00 Collecting idna==3.4 Downloading idna-3.4-py3-none-any.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 392.3 MB/s eta 0:00:00 Collecting importlib-metadata==4.13.0 Downloading importlib_metadata-4.13.0-py3-none-any.whl (23 kB) Collecting iniconfig==2.0.0 Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB) Collecting iso8601==1.1.0 Downloading iso8601-1.1.0-py3-none-any.whl (9.9 kB) Collecting itsdangerous==2.1.2 Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB) Collecting Jinja2==3.1.2 Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 401.7 MB/s eta 0:00:00 Collecting kombu==5.2.4 Downloading kombu-5.2.4-py3-none-any.whl (189 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 189.2/189.2 kB 273.5 MB/s eta 0:00:00 Collecting launchpadlib==1.11.0 Downloading launchpadlib-1.11.0-py2.py3-none-any.whl (217 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 217.7/217.7 kB 292.6 MB/s eta 0:00:00 Collecting lazr.restfulclient==0.14.5 Downloading lazr.restfulclient-0.14.5-py3-none-any.whl (70 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.6/70.6 kB 292.4 MB/s eta 0:00:00 Collecting lazr.uri==1.0.6 Downloading lazr.uri-1.0.6.tar.gz (18 kB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting lxml==4.9.2 Downloading lxml-4.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (7.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 87.3 MB/s eta 0:00:00 Collecting MarkupSafe==2.1.2 Downloading MarkupSafe-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB) Collecting mirakuru==2.4.2 Downloading mirakuru-2.4.2-py3-none-any.whl (29 kB) Collecting msgpack==1.0.4 Downloading msgpack-1.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (316 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.0/317.0 kB 275.4 MB/s eta 0:00:00 Collecting multidict==6.0.4 Downloading multidict-6.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.5/114.5 kB 472.5 MB/s eta 0:00:00 Collecting mypy-extensions==0.4.3 Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB) Collecting oauthlib==3.2.2 Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 460.3 MB/s eta 0:00:00 Collecting packaging==23.0 Downloading packaging-23.0-py3-none-any.whl (42 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.7/42.7 kB 277.0 MB/s eta 0:00:00 Collecting pg8000==1.29.4 Downloading pg8000-1.29.4-py3-none-any.whl (51 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.4/51.4 kB 358.5 MB/s eta 0:00:00 Collecting pika==1.3.1 Downloading pika-1.3.1-py3-none-any.whl (155 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 460.7 MB/s eta 0:00:00 Collecting pkginfo==1.9.6 Downloading pkginfo-1.9.6-py3-none-any.whl (30 kB) Collecting pluggy==1.0.0 Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB) Collecting port-for==0.6.3 Downloading port_for-0.6.3-py3-none-any.whl (21 kB) Collecting prompt-toolkit==3.0.36 Downloading prompt_toolkit-3.0.36-py3-none-any.whl (386 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 386.4/386.4 kB 224.1 MB/s eta 0:00:00 Collecting psutil==5.9.4 Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 280.2/280.2 kB 375.0 MB/s eta 0:00:00 Collecting psycopg2==2.9.5 Downloading psycopg2-2.9.5.tar.gz (384 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 384.3/384.3 kB 176.9 MB/s eta 0:00:00 Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting pycparser==2.21 Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 391.8 MB/s eta 0:00:00 Collecting pyparsing==3.0.9 Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 466.1 MB/s eta 0:00:00 Collecting pytest==7.2.1 Downloading pytest-7.2.1-py3-none-any.whl (317 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 kB 289.6 MB/s eta 0:00:00 Collecting pytest-postgresql==3.1.3 Downloading pytest_postgresql-3.1.3-py3-none-any.whl (41 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.6/41.6 kB 327.9 MB/s eta 0:00:00 Collecting python-dateutil==2.8.2 Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 259.8 MB/s eta 0:00:00 Collecting python-debian==0.1.49 Downloading python_debian-0.1.49-py3-none-any.whl (132 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 435.2 MB/s eta 0:00:00 Collecting python-magic==0.4.27 Downloading python_magic-0.4.27-py2.py3-none-any.whl (13 kB) Collecting python-mimeparse==1.6.0 Downloading python_mimeparse-1.6.0-py2.py3-none-any.whl (6.1 kB) Collecting pytz==2022.7.1 Downloading pytz-2022.7.1-py2.py3-none-any.whl (499 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 499.4/499.4 kB 193.2 MB/s eta 0:00:00 Collecting PyYAML==6.0 Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 682.2/682.2 kB 130.4 MB/s eta 0:00:00 Collecting redis==4.4.2 Downloading redis-4.4.2-py3-none-any.whl (237 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 237.8/237.8 kB 232.8 MB/s eta 0:00:00 Collecting repomd==0.2.1 Downloading repomd-0.2.1-py3-none-any.whl (3.9 kB) Collecting requests==2.28.2 Downloading requests-2.28.2-py3-none-any.whl (62 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 414.2 MB/s eta 0:00:00 Collecting retrying==1.3.4 Downloading retrying-1.3.4-py3-none-any.whl (11 kB) Collecting scramp==1.4.4 Downloading scramp-1.4.4-py3-none-any.whl (13 kB) Collecting sentry-sdk==1.14.0 Downloading sentry_sdk-1.14.0-py2.py3-none-any.whl (178 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.9/178.9 kB 482.8 MB/s eta 0:00:00 Collecting six==1.16.0 Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting sortedcontainers==2.4.0 Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB) Collecting soupsieve==2.3.2.post1 Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB) Collecting swh.core==2.20.0 Downloading swh.core-2.20.0-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 119.2 MB/s eta 0:00:00 Collecting swh.counters==0.9.1 Downloading swh.counters-0.9.1-py3-none-any.whl (37 kB) Collecting swh.journal==1.2.1 Downloading swh.journal-1.2.1-py3-none-any.whl (40 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 kB 362.3 MB/s eta 0:00:00 Collecting swh.lister==4.3.3 Downloading swh.lister-4.3.3-py3-none-any.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 154.4 MB/s eta 0:00:00 Collecting swh.loader.core==5.1.0 Downloading swh.loader.core-5.1.0-py3-none-any.whl (10.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.0/10.0 MB 110.0 MB/s eta 0:00:00 Collecting swh.loader.git==2.1.0 Downloading swh.loader.git-2.1.0-py3-none-any.whl (73 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.8/73.8 kB 422.7 MB/s eta 0:00:00 Collecting swh.loader.metadata==0.0.3 Downloading swh.loader.metadata-0.0.3-py3-none-any.whl (34 kB) Collecting swh.model==6.6.1 Downloading swh.model-6.6.1-py3-none-any.whl (136 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 136.2/136.2 kB 456.0 MB/s eta 0:00:00 Collecting swh.objstorage==2.0.1 Downloading swh.objstorage-2.0.1-py3-none-any.whl (93 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.7/93.7 kB 447.2 MB/s eta 0:00:00 Collecting swh.perfecthash==0.1.2 Downloading swh.perfecthash-0.1.2.tar.gz (27 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Installing backend dependencies: started Installing backend dependencies: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting swh.scheduler==1.5.0 Downloading swh.scheduler-1.5.0-py3-none-any.whl (166 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 kB 460.9 MB/s eta 0:00:00 Collecting swh.storage==1.7.3 Downloading swh.storage-1.7.3-py3-none-any.whl (449 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 449.7/449.7 kB 178.0 MB/s eta 0:00:00 Collecting tenacity==8.1.0 Downloading tenacity-8.1.0-py3-none-any.whl (23 kB) Collecting testing.common.database==2.0.3 Downloading testing.common.database-2.0.3-py2.py3-none-any.whl (10 kB) Collecting testing.postgresql==1.3.0 Downloading testing.postgresql-1.3.0-py2.py3-none-any.whl (8.9 kB) Collecting toml==0.10.2 Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) Collecting tomli==2.0.1 Downloading tomli-2.0.1-py3-none-any.whl (12 kB) Collecting typing_extensions==4.4.0 Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB) Collecting urllib3==1.26.14 Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 kB 461.8 MB/s eta 0:00:00 Collecting vine==5.0.0 Downloading vine-5.0.0-py2.py3-none-any.whl (9.4 kB) Collecting wadllib==1.3.6 Downloading wadllib-1.3.6.tar.gz (62 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.2/62.2 kB 357.8 MB/s eta 0:00:00 Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting wcwidth==0.2.6 Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB) Collecting Werkzeug==2.2.2 Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.7/232.7 kB 286.9 MB/s eta 0:00:00 Collecting wrapt==1.14.1 Downloading wrapt-1.14.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 412.5 MB/s eta 0:00:00 Collecting yarl==1.8.2 Downloading yarl-1.8.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 264.0/264.0 kB 248.9 MB/s eta 0:00:00 Collecting zipp==3.11.0 Downloading zipp-3.11.0-py3-none-any.whl (6.6 kB) Requirement already satisfied: setuptools>=3.0 in /usr/local/lib/python3.10/site-packages (from gunicorn==20.1.0->-r requirements-frozen.txt (line 31)) (65.5.1) Building wheels for collected packages: lazr.uri, psycopg2, swh.perfecthash, wadllib Building wheel for lazr.uri (setup.py): started Building wheel for lazr.uri (setup.py): finished with status 'done' Created wheel for lazr.uri: filename=lazr.uri-1.0.6-py3-none-any.whl size=20362 sha256=95d28b423a964c31846e2703d34644cc65c08c48f7288a7b396764e2362d4678 Stored in directory: /tmp/pip-ephem-wheel-cache-2q9b809_/wheels/a3/09/68/5456b2eb028e206bba5f703f3d1a34e88f51e76f6249e24d94 Building wheel for psycopg2 (setup.py): started Building wheel for psycopg2 (setup.py): finished with status 'done' Created wheel for psycopg2: filename=psycopg2-2.9.5-cp310-cp310-linux_x86_64.whl size=479238 sha256=28ffeef3ac10b3f352559f642eac1978aff82dec22319433988a70912ac48843 Stored in directory: /tmp/pip-ephem-wheel-cache-2q9b809_/wheels/07/2f/1d/00dd98e4de351e52b04ad341a254fe6bebf7c6edb9523ad9a1 Building wheel for swh.perfecthash (pyproject.toml): started Building wheel for swh.perfecthash (pyproject.toml): finished with status 'done' Created wheel for swh.perfecthash: filename=swh.perfecthash-0.1.2-cp310-cp310-linux_x86_64.whl size=47880 sha256=464c7e8844afa32e993401475c71ed7223524b69749f775bb037b3c90ba4bc8a Stored in directory: /tmp/pip-ephem-wheel-cache-2q9b809_/wheels/21/bb/54/e5b93f41dce533d4b7cd25effc1171b986b5dab0f2403f6a7e Building wheel for wadllib (setup.py): started Building wheel for wadllib (setup.py): finished with status 'done' Created wheel for wadllib: filename=wadllib-1.3.6-py3-none-any.whl size=58279 sha256=e58938b592dc63f1aabd67db8c9c9271e79798d9310d39063efec712c7d054d3 Stored in directory: /tmp/pip-ephem-wheel-cache-2q9b809_/wheels/48/01/fd/2f6448c6149fca1c0606c084d83e98d44a1830316af92bfef4 Successfully built lazr.uri psycopg2 swh.perfecthash wadllib Installing collected packages: wcwidth, testing.common.database, sortedcontainers, pytz, python-mimeparse, mypy-extensions, msgpack, confluent-kafka, billiard, asn1crypto, zipp, wrapt, vine, urllib3, typing_extensions, tomli, toml, tenacity, soupsieve, six, scramp, PyYAML, python-magic, pyparsing, pycparser, psycopg2, psutil, prompt-toolkit, port-for, pluggy, pkginfo, pika, packaging, oauthlib, multidict, MarkupSafe, lxml, lazr.uri, itsdangerous, iso8601, iniconfig, idna, humanize, gunicorn, frozenlist, exceptiongroup, distro, defusedxml, click, charset-normalizer, chardet, certifi, blinker, attrs, async-timeout, yarl, Werkzeug, wadllib, sentry-sdk, retrying, requests, repomd, redis, python-debian, python-dateutil, pytest, mirakuru, Jinja2, importlib-metadata, hypothesis, httplib2, geomet, dulwich, Deprecated, click-repl, click-plugins, click-didyoumean, cffi, beautifulsoup4, attrs-strict, amqp, aiosignal, swh.perfecthash, swh.model, swh.core, pytest-postgresql, pg8000, lazr.restfulclient, kombu, Flask, cassandra-driver, aiohttp, testing.postgresql, swh.journal, launchpadlib, celery, aiohttp-utils, swh.objstorage, swh.counters, swh.storage, swh.scheduler, swh.loader.core, swh.lister, swh.loader.metadata, swh.loader.git Successfully installed Deprecated-1.2.13 Flask-2.2.2 Jinja2-3.1.2 MarkupSafe-2.1.2 PyYAML-6.0 Werkzeug-2.2.2 aiohttp-3.8.3 aiohttp-utils-3.1.1 aiosignal-1.3.1 amqp-5.1.1 asn1crypto-1.5.1 async-timeout-4.0.2 attrs-22.2.0 attrs-strict-1.0.0 beautifulsoup4-4.11.1 billiard-3.6.4.0 blinker-1.5 cassandra-driver-3.25.0 celery-5.2.7 certifi-2022.12.7 cffi-1.15.1 chardet-5.1.0 charset-normalizer-2.1.1 click-8.1.3 click-didyoumean-0.3.0 click-plugins-1.1.1 click-repl-0.2.0 confluent-kafka-2.0.2 defusedxml-0.7.1 distro-1.8.0 dulwich-0.21.2 exceptiongroup-1.1.0 frozenlist-1.3.3 geomet-0.2.1.post1 gunicorn-20.1.0 httplib2-0.21.0 humanize-4.4.0 hypothesis-6.65.0 idna-3.4 importlib-metadata-4.13.0 iniconfig-2.0.0 iso8601-1.1.0 itsdangerous-2.1.2 kombu-5.2.4 launchpadlib-1.11.0 lazr.restfulclient-0.14.5 lazr.uri-1.0.6 lxml-4.9.2 mirakuru-2.4.2 msgpack-1.0.4 multidict-6.0.4 mypy-extensions-0.4.3 oauthlib-3.2.2 packaging-23.0 pg8000-1.29.4 pika-1.3.1 pkginfo-1.9.6 pluggy-1.0.0 port-for-0.6.3 prompt-toolkit-3.0.36 psutil-5.9.4 psycopg2-2.9.5 pycparser-2.21 pyparsing-3.0.9 pytest-7.2.1 pytest-postgresql-3.1.3 python-dateutil-2.8.2 python-debian-0.1.49 python-magic-0.4.27 python-mimeparse-1.6.0 pytz-2022.7.1 redis-4.4.2 repomd-0.2.1 requests-2.28.2 retrying-1.3.4 scramp-1.4.4 sentry-sdk-1.14.0 six-1.16.0 sortedcontainers-2.4.0 soupsieve-2.3.2.post1 swh.core-2.20.0 swh.counters-0.9.1 swh.journal-1.2.1 swh.lister-4.3.3 swh.loader.core-5.1.0 swh.loader.git-2.1.0 swh.loader.metadata-0.0.3 swh.model-6.6.1 swh.objstorage-2.0.1 swh.perfecthash-0.1.2 swh.scheduler-1.5.0 swh.storage-1.7.3 tenacity-8.1.0 testing.common.database-2.0.3 testing.postgresql-1.3.0 toml-0.10.2 tomli-2.0.1 typing_extensions-4.4.0 urllib3-1.26.14 vine-5.0.0 wadllib-1.3.6 wcwidth-0.2.6 wrapt-1.14.1 yarl-1.8.2 zipp-3.11.0 Defaulting to user installation because normal site-packages is not writeable Requirement already satisfied: gunicorn in ./.local/lib/python3.10/site-packages (20.1.0) Requirement already satisfied: setuptools>=3.0 in /usr/local/lib/python3.10/site-packages (from gunicorn) (65.5.1) Removing intermediate container 7c09ab0c3117 ---> 9d94f6f5877e Step 10/15 : ENV SWH_CONFIG_FILENAME=/etc/swh/config.yml ---> Running in b91e68bbedc9 Removing intermediate container b91e68bbedc9 ---> e1e2a98179f6 Step 11/15 : ENV SWH_WORKER_INSTANCE=loader ---> Running in 6c3fd3d1d57c Removing intermediate container 6c3fd3d1d57c ---> ef351305264c Step 12/15 : ENV CONCURRENCY=1 ---> Running in 5e00a47dae31 Removing intermediate container 5e00a47dae31 ---> 9ab4288e1edf Step 13/15 : ENV MAX_TASKS_PER_CHILD=1 ---> Running in be8cf81adede Removing intermediate container be8cf81adede ---> ee53c0429f3a Step 14/15 : ENV LOGLEVEL=INFO ---> Running in 4cc3be6e6963 Removing intermediate container 4cc3be6e6963 ---> 28ecf4b825b1 Step 15/15 : ENTRYPOINT "/opt/swh/entrypoint.sh" ---> Running in de9df6e215ba Removing intermediate container de9df6e215ba ---> da151a471177 Successfully built da151a471177 Successfully tagged container-registry.softwareheritage.org/swh/infra/swh-apps/loader_git:20230124.1
Refs. sysadm-environment#4724 (closed)