Requests to the scheduler API take up to 6 seconds in prod
Right now, this command take 2.5s to execute:
echo 'indexer_origin_head;oneshot;[[1]];{"policy_update": "update-dups", "parse_ids": false}' | time python3 -m swh.scheduler.cli --cls remote --url http://saatchi.internal.softwareheritage.org:5008/ task schedule /dev/stdin -c type -c policy -c args -c kwargs -d ';'Yesterday, it was taking 5s. I believe this is because I scheduled a batch of 100k tasks (that is now done); with this command:
python3 -c 'for x in range(100000, 200000): print("""indexer_origin_head;oneshot;[[%d]];{"policy_update": "update-dups", "parse_ids": false}""" % x)' | python3 -m swh.scheduler.cli --cls remote --url http://saatchi.internal.softwareheritage.org:5008/ task schedule /dev/stdin -c type -c policy -c args -c kwargs -d ';'Migrated from T1492 (view on Phabricator)