Staging instance, all changes can be removed at any time

Skip to content

scheduler.backend_es: Leave index opened when streaming bulk

Antoine R. Dumont requested to merge generated-differential-D2614-source into generated-differential-D2614-target

Prior to this commit, we had the proper behavior of closing index when done streaming. Unfortunately, this created too much gc on es nodes down the line. So for now, we remove that behavior.

Note that this implies we need another cog that makes a pass once in a while on indices to close.

Also, this has been running on production for approximately 1 or 2 weeks now and no more gc issues arose since then.

Test Plan

tox

Migrated from D2614 (view on Phabricator)

Merge request reports