Staging instance, all changes can be removed at any time

Skip to content

Don't materialize vec of values while building MPHs

vlorentz requested to merge lazy-par-mph into master

Instead, pthash will iterate on the values (while we read them from disk) and only materialize a vec of hashes, which is much smaller than a vec of values.

When building the SWHIDs MPH, this reduces memory usage from 3.6 to 0.6TB.

Merge request reports