Refactor and deduplicate HTTP requests code in listers
Closed
requested to merge generated-differential-D8520-source into generated-differential-D8520-target
Staging instance, all changes can be removed at any time
Numerous listers were using the same page_request method or equivalent in their implementation so prefer to deduplicate that code by adding an http_request method in base lister class: swh.lister.pattern.Lister.
That method simply wraps a call to requests.Session.request and logs some useful info for debugging and error reporting, also an HTTPError will be raised if a request ends up with an error.
All listers using that new method now benefit of requests retry when an HTTP error occurs thanks to the use of the http_retry decorator.
Depends on !310 (closed)
Migrated from D8520 (view on Phabricator)