Staging instance, all changes can be removed at any time

Skip to content

package/utils: Prevent content automatic deflate in download function

Antoine Lambert requested to merge anlambert/swh-loader-core:download-prevent-auto-deflate into master

The python requests library automatically deflate downloaded content bytes if the response header content-encoding is set to a supported encoding.

This behavior can make a file checksum check after download fail as the expected checksum was computed on the compressed version of the file, not the uncompressed one.

So ensure to prevent automatic deflate by reading response raw content instead of using the iter_content method.

This should fix a lot of events in SWH-LOADER-CORE-1DT.

Merge request reports