Add stagit lister
That list is really near cgit & gitweb implementations. The data is again structured differently though so this implementation on its own.
I did this while I was twiddling my thumbs waiting for automation to finish its stuff. Tests within the docker-dev with various instances are all ok [1].
Refs. swh/meta#5048 (closed)
[1]
$ cat stagit-forges.txt
https://codemadness.org/git/
https://git.acid.vegas/
https://a3nm.net/git/
https://www.brianlane.com/git/
https://git.codemadness.org/
https://git.tronto.net/
https://git.2f30.org/
$ for url in $(cat stagit-forges.txt); do swh-doco exec swh-lister swh lister run -l stagit url="$url"; done
+ cd /home/tony/work/inria/repo/swh/swh-environment/docker
+ docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-lister swh lister run -l stagit url=https://codemadness.org/git/
+ cd /home/tony/work/inria/repo/swh/swh-environment/docker
+ docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-lister swh lister run -l stagit url=https://git.acid.vegas/
+ cd /home/tony/work/inria/repo/swh/swh-environment/docker
+ docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-lister swh lister run -l stagit url=https://a3nm.net/git/
+ cd /home/tony/work/inria/repo/swh/swh-environment/docker
+ docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-lister swh lister run -l stagit url=https://www.brianlane.com/git/
+ cd /home/tony/work/inria/repo/swh/swh-environment/docker
+ docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-lister swh lister run -l stagit url=https://git.codemadness.org/
+ cd /home/tony/work/inria/repo/swh/swh-environment/docker
+ docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-lister swh lister run -l stagit url=https://git.tronto.net/
+ cd /home/tony/work/inria/repo/swh/swh-environment/docker
+ docker-compose -f docker-compose.yml -f docker-compose.override.yml exec swh-lister swh lister run -l stagit url=https://git.2f30.org/
WARNING:swh.lister.pattern:Unexpected HTTP status code 404 on https://git.2f30.org//aoc/log.html: b'<!DOCTYPE html>\n<html>\n<head>\n<meta charset="utf-8">\n<title>404 Not Found</title>\n<style type="text/css"><!--\nbody { background-color: white; color: black; font-family: \'Comic Sans MS\', \'Chalkboard SE\', \'Comic Neue\', sans-serif; }\nhr { border: 0; border-bottom: 1px dashed; }\n@media (prefers-color-scheme: dark) {\nbody { background-color: #1E1F21; color: #EEEFF1; }\na { color: #BAD7FF; }\n}\n--></style>\n</head>\n<body>\n<h1>404 Not Found</h1>\n<hr>\n<address>OpenBSD httpd</address>\n</body>\n</html>\n'
WARNING:swh.lister.stagit.lister:Unexpected HTTP status code 404 on https://git.2f30.org//aoc/log.html
$ psql service=swh-scheduler-dev # <- connect to the scheduler db in docker
2023-07-11 18:51:14 swh-scheduler@localhost:5433 λ select instance_name, count(*) from listed_origins lo inner join listers l on l.id=lo.lister_id where lister_id in (select id from listers where name='stagit') group by instance_name;
+---------------------+-------+
| instance_name | count |
+---------------------+-------+
| a3nm.net | 49 |
| codemadness.org | 43 |
| git.2f30.org | 82 |
| git.codemadness.org | 43 |
| git.tronto.net | 19 |
| www.brianlane.com | 15 |
+---------------------+-------+
(6 rows)
Time: 4.115 mscc @guillaume ;p
Edited by Antoine R. Dumont