Staging instance, all changes can be removed at any time

Skip to content

Added level of abstraction in indexers to use BaseIndexer for revisions

Morane Otilia Gruenpeter requested to merge generated-differential-D233-source into generated-differential-D233-target
  • renaming methods filter_contents to filter and index_content to index in all sub-classes and orchestrator
  • renaming dependencies to ContentIndexer instead of BaseIndexer
  • renaming in tests

Added RevisionMetadataIndexer with a detection tool for metadata

  • RevisionMetadataIndexer takes a list of revisions and detects in the root directory all the file names supported by the swh-metadata-detector version 0.0.1 that can contain metadata

  • checks if files where translated before in the content_metadata table

  • if not: sends the files to indexation

  • aggregates results

resolves T738

Note: should keep results in revision_metadata but this part is not ready in the storage

  • also, changed init of ContentMetadataIndexer with tool in args

Updated documentation with new revision indexer

Test Plan

testing swh-metadata-detector and RevisionMetadataIndexer with MockStorage

Migrated from D233 (view on Phabricator)

Merge request reports