Add support for compressing the graph with only some node types
Typically, this will be used to compress with 'ori,snp,rel,rev' as object types, producing a much smaller graph while still being useful for some experiments
Sorry, it's a big one but it would be pretty hard to test without updating the whole pipeline to support it
There are five kinds of changes:
- Making Python/Luigi workflow pass
--allow-types
to Java - Updating Java
main
functions to take a--allow-types
argument, and pass it toORCGraphDataset
- Make
ORCGraphDataset
supportAllowedNodeTypes
, by simply returningnull
instead of tables, when tables don't match - Updating the rest of the code for skipping some blocks when a table is
null
- Update sanity checks in the Luigi workflow to support missing objects
This is the code used to resolve #4765 (closed)
Depends on !238 (merged), !235 (merged), !237 (merged)
Edited by vlorentz