staging instance, all data can be removed at all time

Skip to content

metadata-loader: Buffer writes to the storage

vlorentz requested to merge vlorentz/swh-charts:metadata-loader-buffer into production

https://grafana.softwareheritage.org/goto/Ih3HTVw4z?orgId=1 shows that 15% of its runtime is spent sending (individual) 'raw_extrinsic_metadata_add' requests, which take ~65ms each.

Found while investigating https://gitlab.softwareheritage.org/swh/meta/-/issues/5035, though that's probably not the cause of the main slowness.

Please don't deploy this before https://gitlab.softwareheritage.org/swh/devel/swh-loader-metadata/-/merge_requests/16 or it would make the metrics "miss" the actual time spent writing

Edited by vlorentz

Merge request reports