production: Deploy golang stack

It's a lister and a loader.

production:

• Packages (swh.lister & swh.loader)
• swh/infra/ci-cd/swh-charts!104 (closed): swh-charts: Add pods for lister and loader
• saatchi: Register golang lister and loader task in the scheduler db [1]
• Add golang lister instance
• Checks
  • listing ok [2]
  • ingestion ongoing [3]

Refs. swh/meta#4124 (closed)

[1] Already there:

swhscheduler@saatchi:~$ swh scheduler --config-file /etc/softwareheritage/scheduler/backend.yml task-type register 2>&1 | grep -i "golang"
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin loader.golang
INFO:swh.scheduler.cli.task_type:Loading entrypoint for plugin lister.golang

[2]

swhscheduler@saatchi:~$ swh scheduler --url http://saatchi.internal.softwareheritage.org:5008/    task add list-golang-incremental
Created 1 tasks

Task 415617076
  Next run: today (2023-07-27T14:34:08.004225+00:00)
  Interval: 1 day, 0:00:00
  Type: list-golang-incremental
  Policy: recurring
  Args:
  Keyword args:

...

2023-07-27 16:40:32 softwareheritage-scheduler@belvedere:5432 λ select instance_name, count(*) from listed_origins lo inner join listers l on l.id=lo.lister_id where lister_id in (select id from listers where name='golang') group by instance_name order by count asc;
+---------------+-------+
| instance_name | count |
+---------------+-------+
| golang        | 92877 |
+---------------+-------+
(1 row)

Time: 1934.188 ms (00:01.934)

[3]

root@saatchi:~# journalctl -xef -u swh-scheduler-schedule-recurrent.service  | grep golang
Jul 27 14:37:25 saatchi swh[516385]: INFO:swh.scheduler.celery_backend.recurrent_visits:Terminating visit scheduling threads: arch, archive-files, aur, bzr, conda, content, cpan, cran, crates, cvs, deb, deb-package, deposit, directory, git, git-checkout, git-from-dir, git-from-zip, golang, hackage, hex, hg, hg-checkout, hg-from-archive, maven, nixguix, npm, opam, pubdev, puppet, pypi, rpm, rubygems, svn, svn-from-archive, svn-from-remote-dump, tarball-directory
Jul 27 14:39:01 saatchi swh[2074573]: INFO:swh.scheduler.celery_backend.recurrent_visits:Skewed fetch for visit type golang with policy already_visited_order_by_lag: fetched 0.0, requested 0.4
Jul 27 14:39:01 saatchi swh[2074573]: INFO:swh.scheduler.celery_backend.recurrent_visits:Skewed fetch for visit type golang with policy never_visited_oldest_update_first: fetched 1.0, requested 0.4
Jul 27 14:39:01 saatchi swh[2074573]: INFO:swh.scheduler.celery_backend.recurrent_visits:Skewed fetch for visit type golang with policy origins_without_last_update: fetched 0.0, requested 0.2
Jul 27 14:39:04 saatchi swh[2074573]: INFO:swh.scheduler.celery_backend.recurrent_visits:golang: 4000 visits scheduled in queue swh.loader.package.golang.tasks.LoadGolang

...

$ period=84h; kail --context archive-production-rke2 -d loader-golang --since $period | grep "Listed\|succeeded\|Fetched"  | cut -d':' -f2- | jq -r .message
Task swh.loader.package.golang.tasks.LoadGolang[012bf336-914f-4129-9012-b5e3742f51e5] succeeded in 4.583828421775252s: {'status': 'eventful', 'snapshot_id': 'e33ddada7e628c0cc014273faa7158664f894adc'}
Task swh.loader.package.golang.tasks.LoadGolang[1cc65f93-805e-4272-b148-4218dc742c46] succeeded in 2.0068341810256243s: {'status': 'eventful', 'snapshot_id': '76b2a4baa7ae4c4117a6d67cfec52d9aa9c2e008'}
Task swh.loader.package.golang.tasks.LoadGolang[b1c8b905-df9c-4ef6-8748-5069c8e68a73] succeeded in 2.0745020820759237s: {'status': 'eventful', 'snapshot_id': '891b860f81d674dec78ef295a5111e3833f382bb'}