Install the 2 new bare metal servers for cassandra production cluster

Orders: https://mybox.inria.fr/smart-link/13004275-5cee-433b-ba88-3c73141757f0/

cassandra09: Inventory: https://inventory.internal.admin.swh.network/dcim/devices/262/

Summary:

• Management address (DNS): 128.93.134.47 (swh-cas9-adm.inria.fr)
• VLAN configuration: VLAN440
• Internal IP(s): 192.168.100.189
• Internal DNS name(s): cassandra09.internal.softwareheritage.org

cassandra10: Inventory: https://inventory.internal.admin.swh.network/dcim/devices/263/

Summary:

• Management address (DNS): 128.93.134.48 (swh-cas10-adm.inria.fr)
• VLAN configuration: VLAN440
• Internal IP(s): 192.168.100.190
• Internal DNS name(s): cassandra10.internal.softwareheritage.org

Tasks:

• Declare the servers in the inventory
• Add the management info in the credential store
• Install the OS
• Add puppet configuration (if needed)
• Configure firewall rules (if needed)
• Configure zfs (only one directory for data and commitlog mountpoint => /srv/cassandra/instance1)
• Add the servers one by one in the cluster
  • Launch a full repair on one node and wait for the end
  • Launch a full repair on the other node and wait for the end
  • Launch a cleanup on all the servers except the last one:
    • cassandra01
    • cassandra02
    • cassandra03
    • cassandra04
    • cassandra05
    • cassandra06
    • cassandra07
    • cassandra08
    • cassandra09
• Declare the servers in the prometheus rules to activate the metrics and monitoring