Deploy first webhooks experiment on staging

Now that swh-webhooks package implements the features we need and a new endpoint dedicated to receive webhook messages was added to swh-web, we can deploy the experiment for updating Save Code Now request statuses using webhooks in staging.

The following is required to deploy the webhooks experiment:

• Add a new sentry project (swh-webhooks ?) dedicated to receive error reports related to webhooks

• Generate a svix authentication token (using command svix-server jwt generate) the new journal client (see below) will use to communicate with the svix server and store it in a secure place

• Generate a secret that will be used to sign webhook messages in order for the webapp to verify their authenticity and store it in a secure place. Webhook secret must match the following regexp ^whsec_[a-zA-Z0-9+/=]{32,100}$, it can be generated using the following command for instance echo whsec_$(cat /dev/urandom | tr -dc '[:alnum:]' | fold -w ${1:-32} | head -n 1).

• Register default webhook event types and webapp endpoint using the following commands (those are idempotent so they could be executed by the service executing the journal client when initializing it):

$ swh webhooks event-type register-defaults

$ swh webhooks endpoint create origin.visit https://<swh_web_host>/save/origin/visit/webhook/ --secret <webhook_secret>

• Update webapp configuration file by adding the webhook secret as followed

save_code_now_webhook_secret: <webhook_secret>

• Disable the swh-web cron job that updates Save Code Now requests statuses in a pull manner

• Setup a new journal client service that will receive kafka messages from the origin_visit_status topic and forward them using webhook messages. The service should execute the following command swh webhooks journal-client, configure sentry reports using the SWH_SENTRY_* environment variables and it should have the following configuration file:

journal:
  brokers:
    - <kafka_broker_host_01>
    - ...
  group_id: swh.webhooks.save_code_now
  prefix: swh.journal.objects
  object_types:
    - origin_visit_status
  auto_offset_reset: latest

webhooks:
  svix:
    server_url: <svix_server_url>
    auth_token: <svix_auth_token>

Optionally, the following actions could also be performed:

• Generate another svix authentication token and store it in the credentials repository under //operations/svix. Its purpose is to use the swh-webhooks CLI for administration or checking sent messages for instance. Nevertheless this requires to update firewall rules in order for the svix server to be reachable from the SWH VPN.

• Make the save/origin/visit/webhook endpoint of the webapp not reachable from the wild by updating reverse proxy configuration as it is only for internal use. Nevertheless, the endpoint processing is protected by checking webhook message signatures are valid.

A similar deployment was done in the docker environment and can be found in this commit.