Add a what is needed for mirrors to watch and act on removal notifications
On removals, we will publish notification on a dedicated Kafka topic. This new notification watcher that should be run on mirrors will listen to the topic and in order:
- Create a new masking request in the masking proxy database and add a “decision pending” entry for every SWHID listed in the notification.
- Send an email to the mirror operator with commands to either propagate the removal, mask permanently, or unmask all objects.
See the newly added section in usage.rst
for a description of the new
commands.
Implementation-wise, the passing of information between the mirror
notification watcher and the handle-removal-notification
command
group is done via the masking database. Useful information that
were in the removal notification are serialized in YAML to the masking
request “reason” field. This will then be retrieved when handling
the removal notification, as well as the list of objects that have been
masked. While this is a bit of abuse, it feels way simpler that
creating a whole new database just for this purpose.
We surely will want to revisit this design once the takedown request processing workflow has been implemented.
Address #20