Staging instance, all changes can be removed at any time

Skip to content

Add integration tests for swh-alter

This shall go together with changes in swh-alter!6 (merged)

To test the ability of swh-alter to remove objects from multiple storage, we need to set up a dedicated environment with – in addition to the common PostgreSQL storage – a Cassandra storage fed by a replayer.

swh-alter also needs to get information from swh-graph but the compression is currently too fragile for us to rely on the dedicated container. Instead, we implement in alter_companion.py a trivial mock HTTP server that will always return 404 — making swh-alter fallback on information from storage.

This server is run in a dedicated swh-alter image. This image will also be used to run commands from the same alter_companion.py that perform queries to the PostgreSQL and Cassandra databases to check the absence (or the presence) of a set of SWHIDs.

We can then implement a simple integration scenario where we load two origins, check the presence in the storages, remove one of them, check the absence in the storages, restore the recovery bundle, and check the renewed presence in the storages.

This will later be expanded to check for removal from objstorage(s), Kafka, and Elasticsearch.

Edited by Jérémy Bobbio (Lunar)

Merge request reports

Loading