Staging instance, all changes can be removed at any time

Skip to content

swh-web: Migrate from sqlite to postgresql

This adapts the production settings to allow the use of the postgresql backend.

This does not touch the development or testing settings module though.

Related to swh/infra/sysadm-environment#2945 (closed)

Test Plan

docker tryouts.

Checked migration part is running with the django-admin-migrate dumpdata/loaddata stanza.

Out of the retrieved staging webapp json dump:

$ swh-doco exec swh-web /bin/bash
swh@3f80177ef10a:/$ django-admin loaddata /webapp-schema.staging.json
Installed 132 object(s) from 1 fixture(s)

psql service=swh-web-dev
swh-web=# select * from save_origin_request;
swh-web=# \conninfo
You are connected to database "swh-web" as user "postgres" on host "localhost" (address "::1") at port "5437".
 id |        request_date        | visit_type |                             origin_url                              |  status  | loading_task_id |         visit_date         | loading_task_status
----+----------------------------+------------+---------------------------------------------------------------------+----------+-----------------+----------------------------+---------------------
  1 | 2020-10-30 18:44:05.807+00 | git        | https://github.com/kubernetes/kubernetes.git                        | accepted |               1 | 2020-10-30 22:49:34.565+00 | succeeded
  2 | 2020-10-30 18:55:31.97+00  | git        | https://gitlab.com/ardumont/home                                    | accepted |               2 | 2020-10-30 19:44:19.999+00 | succeeded
  3 | 2020-10-30 18:55:42.809+00 | git        | https://github.com/NixOS/nixpkgs                                    | accepted |               3 | 2020-10-30 19:45:27.076+00 | failed
  4 | 2020-10-30 18:59:10.378+00 | hg         | https://hg.sr.ht/~zimoun/hello-example                              | accepted |              10 | 2020-11-02 14:34:22.766+00 | failed
  5 | 2020-10-31 10:10:12.777+00 | git        | https://github.com/torvalds/linux                                   | accepted |               4 | 2020-11-01 03:02:47.823+00 | failed
  6 | 2020-11-02 08:33:41.303+00 | git        | https://github.com/videolan/vlc                                     | accepted |               5 | 2020-11-02 09:44:37.583+00 | succeeded
  7 | 2020-11-02 09:54:45.282+00 | git        | https://github.com/torvalds/linux                                   | accepted |               6 |                            | failed
  8 | 2020-11-02 13:53:20.516+00 | git        | https://github.com/torvalds/linux                                   | accepted |               7 |                            | failed
  9 | 2020-11-02 14:01:46.448+00 | git        | https://github.com/torvalds/linux                                   | accepted |               8 | 2020-11-02 16:37:06.489+00 | succeeded
 10 | 2020-11-02 14:03:34.946+00 | git        | https://github.com/NixOS/nixpkgs                                    | accepted |               9 |                            | failed
 11 | 2020-11-02 15:53:18.317+00 | git        | https://github.com/NixOS/nixpkgs                                    | accepted |              12 | 2020-11-03 00:14:12.155+00 | succeeded
 12 | 2020-11-02 17:12:48.265+00 | git        | https://gitorious.org/parmap/parmap.git                             | accepted |              13 | 2020-11-02 17:13:34.731+00 | failed
 13 | 2020-11-03 09:40:50.622+00 | git        | https://github.com/vsellier/easy-cozy                               | accepted |           34059 | 2020-11-03 09:40:56.756+00 | succeeded
 14 | 2020-11-03 22:38:27.531+00 | git        | https://github.com/kubernetes/kubernetes.git                        | accepted |           34069 | 2020-11-03 22:49:50.343+00 | succeeded
 15 | 2020-11-03 22:40:01.333+00 | git        | https://github.com/torvalds/linux                                   | accepted |           34070 | 2020-11-04 00:57:04.078+00 | succeeded
 16 | 2020-11-04 11:08:01.602+00 | git        | https://forge.softwareheritage.org/source/swh-core.git              | pending  |              -1 |                            | not created
 17 | 2020-11-10 08:15:04.852+00 | git        | https://github.com/videolan/vlc                                     | accepted |           34073 | 2020-11-10 08:28:04.666+00 | succeeded

production like success as well but a bit more tedious:

swh@c32204adf7fa:/$ django-admin loaddata --settings=swh.web.settings.production /swh-web.production.json
Installed 71460 object(s) from 1 fixture(s)

In docker-compose.override.yaml

...
  swh-web:
    volumes:
      - "$SWH_ENVIRONMENT_HOME/swh-web:/src/swh-web"
      - "/tmp/webapp-schema.staging.json:/webapp-schema.staging.json:ro"
      - "/tmp/swh-web.production.json:/swh-web.production.json:ro"

Migrated from D5392 (view on Phabricator)

Merge request reports

Loading