Merge requests · Platform / Development / swh-dataset · GitLab

Staging instance, all changes can be removed at any time

Silent mode is enabled

All outbound communications are blocked. Learn more.

Delay the unsubscribe to the end of handle_messages

!46 · created Mar 29, 2022 by David Douard generated-differential-D7463-target
- Closed
- 10
- Approved
updated Mar 30, 2022
Move exporter config entries in dedicated sections

!45 · created Mar 29, 2022 by David Douard generated-differential-D7462-target
- Closed
- 7
- Approved
updated Mar 30, 2022
Add support for limited row numbers in ORC files

!44 · created Mar 29, 2022 by David Douard generated-differential-D7461-target
- Closed
- 13
- Approved
updated Mar 30, 2022
Add the raw_manifest column for revision, release and directory ORC files

!43 · created Mar 18, 2022 by David Douard generated-differential-D7389-target
- Closed
- 13
- Approved
updated Mar 29, 2022
Export revision extra headers in a dedicated ORC file

!42 · created Mar 18, 2022 by David Douard generated-differential-D7388-target
- Closed
- 16
- Approved
updated Mar 29, 2022
Add the type fields for revision and origin_visit_status ORC table

!41 · created Mar 18, 2022 by David Douard generated-differential-D7387-target
- Closed
- 14
- Approved
updated Mar 29, 2022
Use the same 'id' column name everywhere in ORC files

!40 · created Mar 18, 2022 by David Douard generated-differential-D7386-target
- Closed
- 14
- Approved
updated Mar 29, 2022
Write related ORC files in the same directory using the same UUID

!39 · created Mar 18, 2022 by David Douard generated-differential-D7385-target
- Closed
- 15
- Approved
updated Mar 29, 2022
Add some user metadata in generated ORC files

!38 · created Mar 18, 2022 by David Douard generated-differential-D7384-target
- Closed
- 16
- Approved
updated Mar 29, 2022
Implement test_orc exporter as a simple function instead of a fixture

!37 · created Mar 18, 2022 by David Douard generated-differential-D7383-target
- Closed
- 14
- Approved
updated Mar 29, 2022
Make the kafka group_id prefix configurable in the config file

!36 · created Mar 18, 2022 by David Douard generated-differential-D7382-target
- Closed
- 13
- Approved
updated Mar 29, 2022
Use a named logger for journalprocessor.py

!35 · created Mar 18, 2022 by David Douard generated-differential-D7381-target
- Closed
- 13
- Approved
updated Mar 29, 2022
Update JournalClientOffsetRanges for swh.journal 0.9

!34 · created Mar 18, 2022 by David Douard generated-differential-D7380-target
- Closed
- 13
- Approved
updated Mar 29, 2022
Encode TimestampWithTimezone as (sec, usec, offset) in ORC file

!33 · created Mar 18, 2022 by David Douard generated-differential-D7379-target
- Closed
- 19
- Approved
updated Mar 29, 2022
Exporters: add option to write in a deterministic location

!32 · created Mar 08, 2022 by Antoine Pietri generated-differential-D7322-target
- Closed
- 8
- Approved
updated Mar 29, 2022
ORC exporter: use ZST compression

!31 · created Jan 25, 2022 by Antoine Pietri generated-differential-D7038-target
- Closed
- 3
- Approved
updated Jan 26, 2022
Add a command to generate a subdataset from a list of SWHIDs using S3

!30 · created Jan 21, 2022 by Antoine Pietri generated-differential-D7010-target
- Closed
- 11
- Approved
updated Jan 07, 2023
journalprocessor: Reuse the Kafka key instead of computing a new one

!29 · created Dec 07, 2021 by vlorentz generated-differential-D6773-target
- Closed
- 7
- Approved
updated Jan 20, 2022
Commit kafka messages which offset has reach the high limit

!28 · created Sep 13, 2021 by David Douard generated-differential-D6247-target
- Closed
- 4
- Approved
updated Sep 14, 2021
Add a JournalClientOffsetRanges.unsubscribe() method

!27 · created Sep 13, 2021 by David Douard generated-differential-D6246-target
- Closed
- 7
- Approved
updated Sep 14, 2021

Prev
1
2
3
4
5
6
7
Next