Delft Solutions - User contributions [en]

Install a new Disk in Server

2024-09-16T09:04:28Z

Tannaz:

== How to Install a New Disk in a Server ==
To insert the disk into the server used for Ceph and configure it for use, including making it a part of the Ceph cluster with Proxmox, follow these steps:

=== Pick a slot number where you want to put the disk ===
* You can find the empty slot numbers or replace a broken disk.
* Check the drive bay number (that is written in front side of the server) to be matched to the slot number in iDRAC
=== Physically Install the disk ===
* Assemble the caddy and insert the disk.
* For hot-swappable drives (including NVMe and non-NVMe), no need to power down the server. You can insert the disk.
* For non-hot-swappable drives, check the server manual to determine if powering off the server is necessary.
* Follow the server’s manual and insert the disk into the correct drive bay, matching the slot number in iDRAC.
=== Create OSD for the new disk ===
* Depends on the server configuration, when you insert the disk in the server, it could automatically be available in Disk Overview on Proxmox and labelled Unused
* Create the OSD with encryption enabled and correct device class (HDD, SSD, NVMe)
* If the OSD automatically didn’t create, you can create an encrypted OSD manually using ```pveceph osd create /dev/<dev> --crush-device-class <dev-class> -–encrypted 1``` command or you can create the OSD in Proxmox web GUI
* Verify it with using ```ceph osd tree``` to integrate the disk into your Ceph cluster
=== Verify OSD in up | in state in Ceph ===
* Checking Ceph's Health by running ceph -s to check the health of the Ceph cluster
* Verify the new OSD is up and running using ceph osd status and check for any errors with ceph health detail
=== Make sure Ceph is healthy and is using the new OSD ===
* Ceph should be healthy and shows you a green state without any remapped PG's, or degraded data redundancy

Resourse: https://pve.proxmox.com/pve-docs/pveceph.1.html

Install a new Disk in Server

2024-09-16T09:02:49Z

Tannaz:

Install a new Disk in Server

2024-09-13T15:52:22Z

Tannaz: Created page with "== How to Install a New Disk in a Server == To insert the disk into the server used for Ceph and configure it for use, including making it a part of the Ceph cluster with Proxmox, follow these steps: Pick a slot number where you want to put the disk You can find the empty slot numbers or replace a broken disk. Check the drive bay number (that is written in front side of the server) to be matched to the slot number in iDRAC Physically Install the disk: Assemble the cadd..."

== How to Install a New Disk in a Server ==
To insert the disk into the server used for Ceph and configure it for use, including making it a part of the Ceph cluster with Proxmox, follow these steps:

Pick a slot number where you want to put the disk
You can find the empty slot numbers or replace a broken disk. Check the drive bay number (that is written in front side of the server) to be matched to the slot number in iDRAC
Physically Install the disk:
Assemble the caddy and insert the disk.
For hot-swappable drives (including NVMe and non-NVMe), no need to power down the server. You can insert the disk.
For non-hot-swappable drives, check the server manual to determine if powering off the server is necessary.
Follow the server’s manual and insert the disk into the correct drive bay, matching the slot number in iDRAC.
Create OSD for the new disk
Depends on the server configuration, when you insert the disk in the server, it could automatically be available in Disk Overview on Proxmox and labelled Unused
Create the OSD with encryption enabled and correct device class (HDD, SSD, NVMe)
If the OSD automatically didn’t create, you can create an encrypted OSD manually using pveceph osd create /dev/<dev> --crush-device-class <dev-class> -–encrypted 1 command or you can create the OSD in Proxmox web GUI
Verify it with using ceph osd tree to integrate the disk into your Ceph cluster
Verify OSD in up | in state in Ceph
Checking Ceph's Health by running ceph -s to check the health of the Ceph cluster
Verify the new OSD is up and running using ceph osd status and check for any errors with ceph health detail
Make sure Ceph is healthy and is using the new OSD
Ceph should be healthy and shows you a green state without any remapped PG's, or degraded data redundancy

Internal

2024-09-13T15:50:08Z

Tannaz:

== Finance ==

=== Exact ===

* [[booking bonus|Booking bonus]]
* [[booking wages|Booking wages]]
* [[booking quarterly hosting invoice|Booking quarterly hosting invoice]]
* [[new receipt|Enter a new receipt]]
* [[reconciliation|Reconciliation of transaction]]
* [[invoicing|Send an invoice]]
* [[payment reminders|Send payment reminder]]
* [[invoice approval|Process for approving invoices (/filed receipts)]]

=== Bunq ===

* [[top up account|Top up expense account]]

== Work Process ==

* [[Definition of done|Definition of Done]]
* [[Incident Handling|Incident Handling]]
* [[SRE Maintenance|SRE Maintenance]]

== Internal Process ==
* [[timetracking|Timetracking process]]
* [[Starting work for a new client]]
* [[12 percent|12% time]]
* [[Annual leave|Annual leave]]
* [[Bonus allocation|Bonus allocation]]
* [[Calamity leave|Calamity leave]]
* [[Overtime|Overtime]]
* [[Retrospectives|Retrospectives]]
* [[Sick leave|Sick leave]]
* [[Training and self-study|Training and Self-Study]]
* [[Daily|Daily]]

== Projects ==

* Era Inventory [[project_era_inventory_api|API Description]]

== SRE ==

''To be further populated with guide from drive''
* [[create gitlab runner host|Create a GitLab runner host]]
* [[vm setup|Create a (Debian) VM]]
* [[border update|Process for updating a border]]
* [[border reboot|Reboot border without downtime]]
* [[WS Proxmox node reboot|Reboot WS Proxmox node without downtime]]
* [[Resize VM Disk]]
* [[SRE tools]]
* [[Enroll Mac in Kerberos]]
* [[Creating a VM on Hetzner]]
* [[Rebooting VM]]
* [[Rebooting Offsite]]
* [[ssh-fingerprints|Verifying SSH fingerprints]]
* [[Removing VM]]
* [[Install a new Disk in Server]]

== Other ==

* [[stack|Greenfield stack]]
* [[standard tools|Standard Tools]]
* [[list of unfurl debuggers|List of unfurl debuggers]]
* [[Recommended suppliers]]

Internal

2024-09-03T12:10:39Z

Tannaz:

== Finance ==

=== Exact ===

* [[booking bonus|Booking bonus]]
* [[booking wages|Booking wages]]
* [[booking quarterly hosting invoice|Booking quarterly hosting invoice]]
* [[new receipt|Enter a new receipt]]
* [[reconciliation|Reconciliation of transaction]]
* [[invoicing|Send an invoice]]
* [[payment reminders|Send payment reminder]]
* [[invoice approval|Process for approving invoices (/filed receipts)]]

=== Bunq ===

* [[top up account|Top up expense account]]

== Work Process ==

* [[Definition of done|Definition of Done]]
* [[Incident Handling|Incident Handling]]
* [[SRE Maintenance|SRE Maintenance]]

== Internal Process ==
* [[timetracking|Timetracking process]]
* [[Starting work for a new client]]
* [[12 percent|12% time]]
* [[Annual leave|Annual leave]]
* [[Bonus allocation|Bonus allocation]]
* [[Calamity leave|Calamity leave]]
* [[Overtime|Overtime]]
* [[Retrospectives|Retrospectives]]
* [[Sick leave|Sick leave]]
* [[Training and self-study|Training and Self-Study]]
* [[Daily|Daily]]

== Projects ==

* Era Inventory [[project_era_inventory_api|API Description]]

== SRE ==

''To be further populated with guide from drive''
* [[create gitlab runner host|Create a GitLab runner host]]
* [[vm setup|Create a (Debian) VM]]
* [[border update|Process for updating a border]]
* [[border reboot|Reboot border without downtime]]
* [[WS Proxmox node reboot|Reboot WS Proxmox node without downtime]]
* [[Resize VM Disk]]
* [[SRE tools]]
* [[Enroll Mac in Kerberos]]
* [[Creating a VM on Hetzner]]
* [[Rebooting VM]]
* [[Rebooting Offsite]]
* [[ssh-fingerprints|Verifying SSH fingerprints]]
* [[Removing VM]]

== Other ==

* [[stack|Greenfield stack]]
* [[standard tools|Standard Tools]]
* [[list of unfurl debuggers|List of unfurl debuggers]]
* [[Recommended suppliers]]

Replace/Install a new SSD in Server

2024-09-03T09:59:58Z

Tannaz:

We use Ceph so RAID is unnecessary for our installment based on these reasons:

* Built-in Redundancy: Ceph provides data replication and redundancy across multiple OSDs (Object Storage Daemons) in the cluster. It automatically replicates data to ensure availability, so RAID's redundancy is redundant.
*
* Performance Impact: RAID can introduce additional latency and reduce the performance of SSDs. Ceph is designed to work efficiently with raw disks, and adding RAID can slow down operations.
*
* Wasted Resources: Using RAID means you're dedicating some of your disk capacity to redundancy (like RAID 1 mirroring). Ceph already replicates data across multiple disks or nodes, so this would lead to unnecessary resource usage.
*
* Complexity: RAID adds another layer of complexity that isn't needed. Managing disks individually with Ceph simplifies management and reduces potential points of failure.
*

== How to Install a New SSD ==
=== Physically Install the SSD ===
Create a maintenance mode in Zabbix for the server
Check the slot number of the empty drive bays in iDRAK (or you can check if the SSD is blinking and if it is active or not)
Assemble the caddy to the SSD
Pull out the empty drive bay
Physically Install the SSD

=== Verify the SSD is Recognized by the System ===
Login in to Proxmox and run `lsblk` to list all the SSDs and verify that SSD is recognized

=== Prepare the SSD for Ceph ===
Install necessary packages
```apt-get update
apt-get install ceph ceph-osd```
Create and Add the OSD: `pveceph createosd /dev/sdX`
Use command `ceph osd tree` to make sure the new OSD is added
Check the Ceph status: `ceph -s`

=== Change the Raid configuration ===

Most people change the raid config from within the OS, then you don't need downtime. We don't have the configuration utility installed so we generally either use the iDRAC or the boot utility
We usually need configure the RAID through the iDRAC or the boot utility.

=== This is what we did before ===
Before reboting the server in Proxmox, you need to create OSD and check 'ceph pg dump | grep remapped' to make sure there is no issue (no remapped PG) and we have multiple copies in Scorpion and we can proceed to rebooting the server
Reboot the server to put the SSD in Non-Raid (we did it but I am not sure it is a good way)
After rebooting we expect the SSD is detected, we can check in iDRAK manager, if the SSD is blinking
Go to the lifecycle controller and boot the server, we expect to see the SSD in raid controller
Check if the SSD shows up in the system diagnostics and make sure the SSD is being recognized by the server
Make sure ceph is healthy

Replace/Install a new SSD in Server

2024-09-02T16:13:00Z

Tannaz:

We use Ceph so RAID is unnecessary for our installment based on these reasons:

* Built-in Redundancy: Ceph provides data replication and redundancy across multiple OSDs (Object Storage Daemons) in the cluster. It automatically replicates data to ensure availability, so RAID's redundancy is redundant.
*
* Performance Impact: RAID can introduce additional latency and reduce the performance of SSDs. Ceph is designed to work efficiently with raw disks, and adding RAID can slow down operations.
*
* Wasted Resources: Using RAID means you're dedicating some of your disk capacity to redundancy (like RAID 1 mirroring). Ceph already replicates data across multiple disks or nodes, so this would lead to unnecessary resource usage.
*
* Complexity: RAID adds another layer of complexity that isn't needed. Managing disks individually with Ceph simplifies management and reduces potential points of failure.
*

== How to Install a New SSD ==
=== Physically Install the SSD ===
Create a maintenance mode in Zabbix for the server
Check the slot number of the empty drive bays in iDRAK (or you can check if the SSD is blinking and if it is active or not)
Assemble the caddy to the SSD
Pull out the empty drive bay
Physically Install the SSD

=== Verify the SSD is Recognized by the System ===
Login in to Proxmox and run `lsblk` to list all the SSDs and verify that SSD is recognized

=== Prepare the SSD for Ceph ===
Install necessary packages
```apt-get update
apt-get install ceph ceph-osd```
Create and Add the OSD: `pveceph createosd /dev/sdX`
Use command `ceph osd tree` to make sure the new OSD is added
Check the Ceph status: `ceph -s`

Before reboting the server in Proxmox, you need to create OSD and check 'ceph pg dump | grep remapped' to make sure there is no issue (no remapped PG) and we have multiple copies in Scorpion and we can proceed to rebooting the server
Reboot the server to put the SSD in Non-Raid (we did it but I am not sure it is a good way)
After rebooting we expect the SSD is detected, we can check in iDRAK manager, if the SSD is blinking
Go to the lifecycle controller and boot the server, we expect to see the SSD in raid controller
Check if the SSD shows up in the system diagnostics and make sure the SSD is being recognized by the server
Make sure ceph is healthy

Replace/Install a new SSD in Server

2024-09-02T16:12:48Z

Tannaz:

We use Ceph so RAID is unnecessary for our installment based on these reasons:

* Built-in Redundancy: Ceph provides data replication and redundancy across multiple OSDs (Object Storage Daemons) in the cluster. It automatically replicates data to ensure availability, so RAID's redundancy is redundant.
*
* Performance Impact: RAID can introduce additional latency and reduce the performance of SSDs. Ceph is designed to work efficiently with raw disks, and adding RAID can slow down operations.
*
* Wasted Resources: Using RAID means you're dedicating some of your disk capacity to redundancy (like RAID 1 mirroring). Ceph already replicates data across multiple disks or nodes, so this would lead to unnecessary resource usage.
*
* Complexity: RAID adds another layer of complexity that isn't needed. Managing disks individually with Ceph simplifies management and reduces potential points of failure.
*

== How to Install a New SSD ==
=== Physically Install the SSD ===
Create a maintenance mode in Zabbix for the server
Check the slot number of the empty drive bays in iDRAK (or you can check if the SSD is blinking and if it is active or not)
Assemble the caddy to the SSD
Pull out the empty drive bay
Physically Install the SSD

=== Verify the SSD is Recognized by the System ===
Login in to Proxmox and run `lsblk` to list all the SSDs and verify that SSD is recognized

==== Prepare the SSD for Ceph ===
Install necessary packages
```apt-get update
apt-get install ceph ceph-osd```
Create and Add the OSD: `pveceph createosd /dev/sdX`
Use command `ceph osd tree` to make sure the new OSD is added
Check the Ceph status: `ceph -s`

Before reboting the server in Proxmox, you need to create OSD and check 'ceph pg dump | grep remapped' to make sure there is no issue (no remapped PG) and we have multiple copies in Scorpion and we can proceed to rebooting the server
Reboot the server to put the SSD in Non-Raid (we did it but I am not sure it is a good way)
After rebooting we expect the SSD is detected, we can check in iDRAK manager, if the SSD is blinking
Go to the lifecycle controller and boot the server, we expect to see the SSD in raid controller
Check if the SSD shows up in the system diagnostics and make sure the SSD is being recognized by the server
Make sure ceph is healthy

Replace/Install a new SSD in Server

2024-09-02T15:14:30Z

Tannaz: Created page with "We use Ceph so RAID is unnecessary for our installment. Ceph provides data replication and redundancy across multiple OSDs (Object Storage Daemons) in the cluster. It automatically replicates data to ensure availability, so RAID's redundancy is redundant."

We use Ceph so RAID is unnecessary for our installment.
Ceph provides data replication and redundancy across multiple OSDs (Object Storage Daemons) in the cluster. It automatically replicates data to ensure availability, so RAID's redundancy is redundant.

Internal

2024-09-02T15:08:22Z

Tannaz:

== Finance ==

=== Exact ===

* [[booking bonus|Booking bonus]]
* [[booking wages|Booking wages]]
* [[booking quarterly hosting invoice|Booking quarterly hosting invoice]]
* [[new receipt|Enter a new receipt]]
* [[reconciliation|Reconciliation of transaction]]
* [[invoicing|Send an invoice]]
* [[payment reminders|Send payment reminder]]
* [[invoice approval|Process for approving invoices (/filed receipts)]]

=== Bunq ===

* [[top up account|Top up expense account]]

== Work Process ==

* [[Definition of done|Definition of Done]]
* [[Incident Handling|Incident Handling]]
* [[SRE Maintenance|SRE Maintenance]]

== Internal Process ==
* [[timetracking|Timetracking process]]
* [[Starting work for a new client]]
* [[12 percent|12% time]]
* [[Annual leave|Annual leave]]
* [[Bonus allocation|Bonus allocation]]
* [[Calamity leave|Calamity leave]]
* [[Overtime|Overtime]]
* [[Retrospectives|Retrospectives]]
* [[Sick leave|Sick leave]]
* [[Training and self-study|Training and Self-Study]]
* [[Daily|Daily]]

== Projects ==

* Era Inventory [[project_era_inventory_api|API Description]]

== SRE ==

''To be further populated with guide from drive''
* [[create gitlab runner host|Create a GitLab runner host]]
* [[vm setup|Create a (Debian) VM]]
* [[border update|Process for updating a border]]
* [[border reboot|Reboot border without downtime]]
* [[WS Proxmox node reboot|Reboot WS Proxmox node without downtime]]
* [[Resize VM Disk]]
* [[SRE tools]]
* [[Enroll Mac in Kerberos]]
* [[Creating a VM on Hetzner]]
* [[Rebooting VM]]
* [[Rebooting Offsite]]
* [[ssh-fingerprints|Verifying SSH fingerprints]]
* [[Removing VM]]
* [[Replace/Install a new SSD in Server]]

== Other ==

* [[stack|Greenfield stack]]
* [[standard tools|Standard Tools]]
* [[list of unfurl debuggers|List of unfurl debuggers]]
* [[Recommended suppliers]]

WS Proxmox node reboot

2024-08-12T13:35:19Z

Tannaz:

== Tips & Notes ==
* If you're expecting to reboot every node in the cluster, do the node with the containers last, to limit the amount of downtime and reboots for them
* Updating a node: `apt update` and `apt full-upgrade`
* Make sure all VMs are actually migratable before adding to a HA group
* If there are containers on the device you are looking to reboot- you are going to need to also create a maintenance mode to cover them (for example teamspeak or awstats)
* Containers will inherit the OS of their host, so you will also need to handle triggers related to their OS updating, where appropriate
== Pre-Work ==
* If a VM or container is going to incur downtime, you must let the affected parties know in advance. Ideally they should be informed the previous day.

== Pre flight checks ==
* Check all Ceph pools are running on at least 3/2 replication
* Check that all running VM's on the node you want to reboot are in HA (if not, add them or migrate them away manually)
* Check that Ceph is healthy -> No remapped PG's, or degraded data redundancy
* You have communicated that downtime is expected to the users who will be affected (Ideally one day in advance)

== Update Process ==
* Update the node: `apt update` and `apt full-upgrade`
* Check the packages that are removed/updated/installed correctly and they are the sane (to make sense)

== Reboot process ==
* Start maintenance mode for the Proxmox node and any containers running on the node
* Start maintenance mode for Ceph, specify that we only want to surpress the trigger for health state being in warning by setting tag `ceph_health` equals `warning`
* Let affected parties know that the mainenance period you told them about in the preflight checks is about to take place.
[[File:Ceph-maintenance.png|thumb]]
* Set noout flag on host: `ceph osd set-group noout <node>`

# gain ssh access to host
# Log in through IPA
# Run the command
* '''Reboot''' node through web GUI
* Wait for node to come back up
* Wait for OSD's to be back online
* Remove noout flag on host: `ceph osd unset-group noout <node>` ,to do this:
* If a kernel update was done, manually execute the `Operating system` item manually to detect the update. Manually executing the two items that indicate a reboot is also usefull if they were firing, to stop them/check no further reboots are needed.
* Ackowledge & close triggers
* Remove maintenance modes

== Aftercare ==
* Ensure that Kaboom API is running on Screwdriver or Paloma. This is to get the best performance for the VM.

WS Proxmox node reboot

2024-08-12T13:28:10Z

Tannaz:

== Tips & Notes ==
* If you're expecting to reboot every node in the cluster, do the node with the containers last, to limit the amount of downtime and reboots for them
* Updating a node: `apt update` and `apt full-upgrade`
* Make sure all VMs are actually migratable before adding to a HA group
* If there are containers on the device you are looking to reboot- you are going to need to also create a maintenance mode to cover them (for example teamspeak or awstats)
* Containers will inherit the OS of their host, so you will also need to handle triggers related to their OS updating, where appropriate
== Pre-Work ==
* If a VM or container is going to incur downtime, you must let the affected parties know in advance. Ideally they should be informed the previous day.

== Pre flight checks ==
* Check all Ceph pools are running on at least 3/2 replication
* Check that all running VM's on the node you want to reboot are in HA (if not, add them or migrate them away manually)
* Check that Ceph is healthy -> No remapped PG's, or degraded data redundancy
* You have communicated that downtime is expected to the users who will be affected (Ideally one day in advance)

== Update Process ==
* Update the node: `apt update` and `apt full-upgrade`
* Check the packages that are removed/updated/installed correctly and they are the sane (to make sense)

== Reboot process ==
* Start maintenance mode for the Proxmox node and any containers running on the node
* Start maintenance mode for Ceph, specify that we only want to surpress the trigger for health state being in warning by setting tag `ceph_health` equals `warning`
* Let affected parties know that the mainenance period you told them about in the preflight checks is about to take place.
[[File:Ceph-maintenance.png|thumb]]
* Set noout flag on host: `ceph osd set-group noout <node>`

# gain ssh access to host
# Log in through IPA
# Run the command

* '''Reboot''' node through web GUI
* Wait for node to come back up
* Wait for OSD's to be back online
* Remove noout flag on host: `ceph osd unset-group noout <node>` ,to do this:

* If a kernel update was done, manually execute the `Operating system` item manually to detect the update. Manually executing the two items that indicate a reboot is also usefull if they were firing, to stop them/check no further reboots are needed.
* Ackowledge & close triggers
* Remove maintenance modes

== Aftercare ==
* Ensure that Kaboom API is running on Screwdriver or Paloma. This is to get the best performance for the VM.

WS Proxmox node reboot

2024-08-12T13:24:29Z

Tannaz:

== Tips & Notes ==
* If you're expecting to reboot every node in the cluster, do the node with the containers last, to limit the amount of downtime and reboots for them
* Updating a node: `apt update` and `apt full-upgrade`
* Make sure all VMs are actually migratable before adding to a HA group
* If there are containers on the device you are looking to reboot- you are going to need to also create a maintenance mode to cover them (for example teamspeak or awstats)
* Containers will inherit the OS of their host, so you will also need to handle triggers related to their OS updating, where appropriate
== Pre-Work ==
* If a VM or container is going to incur downtime, you must let the affected parties know in advance. Ideally they should be informed the previous day.

== Pre flight checks ==
* Check all Ceph pools are running on at least 3/2 replication
* Check that all running VM's on the node you want to reboot are in HA (if not, add them or migrate them away manually)
* Check that Ceph is healthy -> No remapped PG's, or degraded data redundancy
* You have communicated that downtime is expected to the users who will be affected (Ideally one day in advance)

== Update Process ==
* Update the node: `apt update` and `apt full-upgrade`
* Check the packages that are removed/updated/installed correctly and they are the sane (to make sense)

== Reboot process ==
* Start maintenance mode for the Proxmox node and any containers running on the node
* Start maintenance mode for Ceph, specify that we only want to surpress the trigger for health state being in warning by setting tag `ceph_health` equals `warning`
* Let affected parties know that the mainenance period you told them about in the preflight checks is about to take place.
[[File:Ceph-maintenance.png|thumb]]
* Set noout flag on host: `ceph osd set-group noout <node>`
* '''Reboot''' node through web GUI
* Wait for node to come back up
* Wait for OSD's to be back online
* Remove noout flag on host: `ceph osd unset-group noout <node>` ,to do this:

# gain ssh access to host
# Log in through IPA
# Run the command

* If a kernel update was done, manually execute the `Operating system` item manually to detect the update. Manually executing the two items that indicate a reboot is also usefull if they were firing, to stop them/check no further reboots are needed.
* Ackowledge & close triggers
* Remove maintenance modes

== Aftercare ==
* Ensure that Kaboom API is running on Screwdriver or Paloma. This is to get the best performance for the VM.

Rebooting Offsite

2024-08-09T15:28:21Z

Tannaz:

= Pre-flight checks =
* Access to the Web GUI
* Access to the terminal through SSH
* Zulip and Gitlab are running on it, so needed to communicate first on Organisational channel, to make sure everyone has read and agreed on it.

= Rebooting =
# Make sure all the pre flight checks are done
# Make an schedule for doing the rebootng.
# Create a maintenance period on Zabbix with all the vms are running on Offsite and Offsite itself
# Make sure the maintenance is active (yellow icon)
# Update the packages through terminal `apt update` and `apt upgrade`
# Make sure that the packages are removed, upgraded and installed correctly and check them
# Click the reboot button for Offsite in the web GUI
# Make sure offsite is actually shutting down
# Wait till it comes online again
# On Zabbix, go to 'Configuration' -> 'all the vms', and click on 'items' for the rebooted vms.
# Search for 'reboot', select the checkbox for both items and click on 'Execute now'.
# Search for 'operating', select the checkbox for the item that has a trigger that uses it, and select 'Execugte now'.
# Go to 'Monitoring' -> 'Problems'. Check the checkbox for triggers that fired for this host. Expected are '<host> has been restarted' and 'Operating system description has changed'. Scroll to the bottom and click on 'Mass update'. In the modal, check the checkbox for 'close problem', 'acknowledge', and in the 'message' box write the reason the host was rebooted. For example 'host was rebooted due to kernel update'
# Remove the maintenance period from the vm

Rebooting Offsite

2024-08-09T15:21:50Z

Tannaz:

= Pre-flight checks =
* Access to the Web GUI
* Access to the terminal through SSH
* Zulip and Gitlab are running on it, so needed to communicate first on Organisational channel, to make sure everyone has read and agreed on it.

= Rebooting =
# Make sure all the pre flight checks are done
# Make an schedule for doing the rebootng.
# Create a maintenance period on Zabbix with all the vms are running on Offsite and Offsite itself
# Make sure the maintenance is active (yellow icon)
# Update the packages through terminal `apt update` and `apt upgrade`
# Make sure that the packages are removed, upgraded and installed correctly and check them
# Click the reboot button for Offsite in the web GUI
# Check if it is actually shut down
# Wait till it comes online again
# On Zabbix, go to 'Configuration' -> 'all the vms', and click on 'items' for the rebooted vms.
# Search for 'reboot', select the checkbox for both items and click on 'Execute now'.
# Search for 'operating', select the checkbox for the item that has a trigger that uses it, and select 'Execugte now'.
# Go to 'Monitoring' -> 'Problems'. Check the checkbox for triggers that fired for this host. Expected are '<host> has been restarted' and 'Operating system description has changed'. Scroll to the bottom and click on 'Mass update'. In the modal, check the checkbox for 'close problem', 'acknowledge', and in the 'message' box write the reason the host was rebooted. For example 'host was rebooted due to kernel update'
# Remove the maintenance period from the vm

Rebooting Offsite

2024-08-09T15:19:51Z

Tannaz:

= Pre-flight checks =
* Access to the Web GUI
* Access to the terminal through SSH
* Zulip and Gitlab are running on it, so needed to communicate first on Organisational channel, to make sure everyone has read and agreed on it.

= Rebooting =
# Make sure all the pre flight checks are done
# Make an schedule for doing the rebootng.
# Create a maintenance period on Zabbix with all the vms are running on Offsite and Offsite itself
# Make sure the maintenance is active (yellow icon)
# Update the packages through terminal `apt update` and `apt upgrade`
# Make sure that the packages are removed, upgraded and installed correctly and check them
# Click the reboot button for the Offsite in the web GUI
# Check if it is actually shut down
# Wait till it comes online again
# On Zabbix, go to 'Configuration' -> 'all the vms', and click on 'items' for the rebooted vms.
# Search for 'reboot', select the checkbox for both items and click on 'Execute now'.
# Search for 'operating', select the checkbox for the item that has a trigger that uses it, and select 'Execugte now'.
# Go to 'Monitoring' -> 'Problems'. Check the checkbox for triggers that fired for this host. Expected are '<host> has been restarted' and 'Operating system description has changed'. Scroll to the bottom and click on 'Mass update'. In the modal, check the checkbox for 'close problem', 'acknowledge', and in the 'message' box write the reason the host was rebooted. For example 'host was rebooted due to kernel update'
# Remove the maintenance period from the vm

Rebooting Offsite

2024-08-08T11:46:44Z

Tannaz: Created page with "= Pre-flight checks = * Access to the Web GUI * Access to the terminal through SSH * Zulip and Gitlab are running on it, so needed to communicate first on Organisational channel, to make sure everyone has read and agreed on it. = Rebooting = # Make sure all the pre flight checks are done # Make an schedule for doing the rebootng. # Create a maintenance period on Zabbix with all the vms are running on Offsite and Offsite itself # Update the packages through terminal `ap..."

= Pre-flight checks =
* Access to the Web GUI
* Access to the terminal through SSH
* Zulip and Gitlab are running on it, so needed to communicate first on Organisational channel, to make sure everyone has read and agreed on it.

= Rebooting =
# Make sure all the pre flight checks are done
# Make an schedule for doing the rebootng.
# Create a maintenance period on Zabbix with all the vms are running on Offsite and Offsite itself
# Update the packages through terminal `apt update` and `apt upgrade`
# Make sure that the packages are removed, upgraded and installed correctly and check them
# Click the reboot button for the Offsite in the web GUI
# Check if it is actually shut down
# Wait till it comes online again
# On Zabbix, go to 'Configuration' -> 'all the vms', and click on 'items' for the rebooted vms.
# Search for 'reboot', select the checkbox for both items and click on 'Execute now'.
# Search for 'operating', select the checkbox for the item that has a trigger that uses it, and select 'Execugte now'.
# Go to 'Monitoring' -> 'Problems'. Check the checkbox for triggers that fired for this host. Expected are '<host> has been restarted' and 'Operating system description has changed'. Scroll to the bottom and click on 'Mass update'. In the modal, check the checkbox for 'close problem', 'acknowledge', and in the 'message' box write the reason the host was rebooted. For example 'host was rebooted due to kernel update'
# Remove the maintenance period from the vm

Internal

2024-08-08T11:20:06Z

Tannaz:

== Finance ==

=== Exact ===

* [[booking bonus|Booking bonus]]
* [[booking wages|Booking wages]]
* [[booking quarterly hosting invoice|Booking quarterly hosting invoice]]
* [[new receipt|Enter a new receipt]]
* [[reconciliation|Reconciliation of transaction]]
* [[invoicing|Send an invoice]]
* [[payment reminders|Send payment reminder]]
* [[invoice approval|Process for approving invoices (/filed receipts)]]

=== Bunq ===

* [[top up account|Top up expense account]]

== Work Process ==

* [[Definition of done|Definition of Done]]
* [[Incident Handling|Incident Handling]]
* [[SRE Maintenance|SRE Maintenance]]

== Internal Process ==
* [[timetracking|Timetracking process]]
* [[Starting work for a new client]]
* [[12 percent|12% time]]
* [[Annual leave|Annual leave]]
* [[Bonus allocation|Bonus allocation]]
* [[Calamity leave|Calamity leave]]
* [[Overtime|Overtime]]
* [[Retrospectives|Retrospectives]]
* [[Sick leave|Sick leave]]
* [[Training and self-study|Training and Self-Study]]
* [[Daily|Daily]]

== Projects ==

* Era Inventory [[project_era_inventory_api|API Description]]

== SRE ==

''To be further populated with guide from drive''
* [[create gitlab runner host|Create a GitLab runner host]]
* [[vm setup|Create a (Debian) VM]]
* [[border update|Process for updating a border]]
* [[border reboot|Reboot border without downtime]]
* [[WS Proxmox node reboot|Reboot WS Proxmox node without downtime]]
* [[Resize VM Disk]]
* [[SRE tools]]
* [[Enroll Mac in Kerberos]]
* [[Creating a VM on Hetzner]]
* [[Rebooting VM]]
* [[Rebooting Offsite]]

== Other ==

* [[stack|Greenfield stack]]
* [[standard tools|Standard Tools]]
* [[list of unfurl debuggers|List of unfurl debuggers]]
* [[Recommended suppliers]]

Internal

2024-07-17T10:16:04Z

Tannaz:

== Finance ==

=== Exact ===

* [[booking bonus|Booking bonus]]
* [[booking wages|Booking wages]]
* [[booking quarterly hosting invoice|Booking quarterly hosting invoice]]
* [[new receipt|Enter a new receipt]]
* [[reconciliation|Reconciliation of transaction]]
* [[invoicing|Send an invoice]]
* [[payment reminders|Send payment reminder]]
* [[invoice approval|Process for approving invoices (/filed receipts)]]

=== Bunq ===

* [[top up account|Top up expense account]]

== Work Process ==

* [[Definition of done|Definition of Done]]
* [[Incident Handling|Incident Handling]]
* [[SRE Maintenance|SRE Maintenance]]

== Internal Process ==
* [[Starting work for a new client]]
* [[12 percent|12% time]]
* [[Annual leave|Annual leave]]
* [[Bonus allocation|Bonus allocation]]
* [[Calamity leave|Calamity leave]]
* [[Overtime|Overtime]]
* [[Retrospectives|Retrospectives]]
* [[Sick leave|Sick leave]]
* [[Training and self-study|Training and Self-Study]]
* [[Daily|Daily]]

== Projects ==

* Era Inventory [[project_era_inventory_api|API Description]]

== SRE ==

''To be further populated with guide from drive''
* [[create gitlab runner host|Create a GitLab runner host]]
* [[vm setup|Create a (Debian) VM]]
* [[border update|Process for updating a border]]
* [[border reboot|Reboot border without downtime]]
* [[WS Proxmox node reboot|Reboot WS Proxmox node without downtime]]
* [[Resize VM Disk]]
* [[SRE tools]]
* [[Enroll Mac in Kerberos]]
* [[Creating a VM on Hetzner]]
* [[Rebooting VM]]

== Other ==

* [[stack|Greenfield stack]]
* [[standard tools|Standard Tools]]
* [[list of unfurl debuggers|List of unfurl debuggers]]
* [[Recommended suppliers]]

Border reboot

2024-07-15T13:13:51Z

Tannaz:

Note: Throughout this guide <ipv4> and <ipv6> are to be replaced by the correct IP's.

== Preflight checks ==
These checks are to be done on the other border (So the border that will stay online). The commands are to be invoked in `vtysh`.
* Confirm our IPv4 block is announced over BGP with `show ip bgp neighbors <ipv4> advertised-routes`
* Confirm our IPv6 block is announced over BGP with `show bgp neighbors <ipv6> advertised-routes`
* Confirm that the border receives the ROUTED IPv4 routes from the router with `show ip route`
* Confirm that the border received the ROUTED & LAN IPv6 routes from the router with `show ipv6 route`
* Set a maintenance period for the host on Zabbix.
* Post in the Zulip in the relevant topic (incident's topic / 'SRE - General' stream) that the border is going to be rebooted.

== Disabling routing through a border ==
On a border in `vtysh`, update the running configuration by invoking the following:

* config
* router bgp
* neighbor <ipv4> shutdown
* neighbor <ipv6> shutdown
* exit
* router ospf
* no default-information originate
* exit
* router ospf6
* no default-information originate
* exit
* exit
* exit

== Reboot the border ==

* After performing the pre-flight checks and disabling the routing, you can choose to wait until traffic has decreased (e.g. using `bmon` to check bandwidth used on interfaces)
* Execute `reboot` command
* When the border is back online, execute relavant items (system uptime, operating system, reboot required) to ensure these will not activate a trigger after disabling maintenance mode
* If you do not expect any Zabbix alert related to the reboot to be fired, delete the maintenance period

== Troubleshooting ==

Undoing the shutdown of the neighbors can be done by invoking `no neighbor <ipv4>/<ipv6> shutdown` in the `router bgp` part of the configuration.

And the `no default-information originate` can be undone by invoking `default-information originate` in the corect ospf part of the configuration (ospf or ospf6, depending on which one you wish to re-enable).

A reload/restart of the service will also reset to normal configuration.

User:Tannaz

2024-07-04T13:15:52Z

Tannaz: Blanked the page

Creating a VM on Hetzner

2024-07-04T13:15:43Z

Tannaz: Created page with "== Creating a Virtual Server (VM) on Hetzner == === Step 1: Generate an RSA Key Pair === First, you need to create an RSA key pair to have a public key for your server. >> ssh-keygen -t rsa === Step 2: Create an account on Hetzner to get access to their development cloud === === Step 3: Set Up a Virtual Server on Hetzner === Ensure you have access to create a virtual server on Hetzner. Follow the instructions in this article to create a cloud server on the Hetzner Cloud..."

== Creating a Virtual Server (VM) on Hetzner ==
=== Step 1: Generate an RSA Key Pair ===
First, you need to create an RSA key pair to have a public key for your server.
>> ssh-keygen -t rsa

=== Step 2: Create an account on Hetzner to get access to their development cloud ===
=== Step 3: Set Up a Virtual Server on Hetzner ===
Ensure you have access to create a virtual server on Hetzner. Follow the instructions in this article to create a cloud server on the Hetzner Cloud Console: Creating a Server on Hetzner.
=== Step 4: Configure Your Server ===
You will likely need both IPv4 and IPv6 addresses for your VM, so make sure to configure both.
Choose the latest version of Debian as your operating system and complete the configuration based on your needs. Name the server properly using the format <your-name server>.dev.delftsolutions.nl, replacing <your-name> with your actual name server.
Once configured, purchase your virtual server. This will take you to the new virtual server page, where you can see both your IPv4 and IPv6 addresses.
=== Step 5: Update DNS Repository ===
Add your IPv4 and IPv6 addresses to the DNS-basic repository and create a merge request.
For the IPv6 make sure “/64” at the end is replaced by 1 just like other IPv6s in the script. And try to make the format like the other lines.
Make sure
=== Step 6: Generate a Fingerprint ===
Your VM will generate a fingerprint to recognize you. This fingerprint acts as a unique identifier for the server and is stored locally.
Use the following command to add the fingerprint to the DNS system:
>> ssh-keyscan -D <your-virtual-server-IPv4>

This command will generate a fingerprint that you need to add to the DNS-basic repository (make sure to remove any lines starting with a semicolon and replace the first IP to your name, again just like the other lines in there).
=== Step 7: Test Your Virtual Server ===
To ensure your virtual server is working correctly, use these commands:
>> dig @ns1.delftsolutions.nl.
>> dig <your-name-server> @ns1.delftsolutions.nl.
>> dig <your-name-server>

These commands will verify if your name server is functioning properly and if you have access to ns1.delftsolutions.nl. But these commands are fun to do, they are not necessary :)
=== Step 8: SSH into Your Virtual Server ===
Once everything is verified and functioning correctly, you can SSH into your virtual server using the following command:
>> ssh root@<your-name-server>

Replace <your-name-server> with the actual name of your server, using root before your name server means you want to be root to make changes in your virtual server.
When it is your first time to get access to your virtual server it will ask you “Are you sure you want to continue connecting (yes/no/[fingerprint])?” and when you say yes, you will be permanently added to the host, and you need to enter the passphrase of your RSA key pair.
Then boom, you have your virtual server :)

Internal

2024-07-04T13:15:33Z

Tannaz:

== Finance ==

=== Exact ===

* [[booking bonus|Booking bonus]]
* [[booking wages|Booking wages]]
* [[new receipt|Enter a new receipt]]
* [[reconciliation|Reconciliation of transaction]]
* [[invoicing|Send an invoice]]
* [[payment reminders|Send payment reminder]]
* [[invoice approval|Process for approving invoices (/filed receipts)]]

=== Bunq ===

* [[top up account|Top up expense account]]

== Work Process ==

* [[Definition of done|Definition of Done]]
* [[Incident Handling|Incident Handling]]
* [[SRE Maintenance|SRE Maintenance]]

== Internal Process ==
* [[Starting work for a new client]]
* [[12 percent|12% time]]
* [[Annual leave|Annual leave]]
* [[Bonus allocation|Bonus allocation]]
* [[Calamity leave|Calamity leave]]
* [[Overtime|Overtime]]
* [[Retrospectives|Retrospectives]]
* [[Sick leave|Sick leave]]
* [[Training and self-study|Training and Self-Study]]

== Projects ==

* Era Inventory [[project_era_inventory_api|API Description]]

== SRE ==

''To be further populated with guide from drive''
* [[create gitlab runner host|Create a GitLab runner host]]
* [[vm setup|Create a (Debian) VM]]
* [[border reboot|Reboot border without downtime]]
* [[WS Proxmox node reboot|Reboot WS Proxmox node without downtime]]
* [[Resize VM Disk]]
* [[SRE tools]]
* [[Enroll Mac in Kerberos]]
* [[Creating a VM on Hetzner]]

== Other ==

* [[stack|Greenfield stack]]
* [[standard tools|Standard Tools]]
* [[list of unfurl debuggers|List of unfurl debuggers]]
* [[Recommended suppliers]]

User:Tannaz

2024-07-04T13:07:17Z

Tannaz:

Tannaz

2024-07-04T13:06:50Z

Tannaz: Blanked the page

DelftSolutions:Tannaz

2024-07-04T11:48:59Z

Tannaz: Blanked the page

User:Tannaz

2024-07-04T11:48:31Z

Tannaz: Blanked the page

DelftSolutions:Tannaz

2024-07-04T11:45:34Z

Tannaz: Tannaz moved page DelftSolutions:Tannaz to Tannaz

#REDIRECT [[Tannaz]]

Tannaz

2024-07-04T11:45:34Z

Tannaz: Tannaz moved page DelftSolutions:Tannaz to Tannaz

#REDIRECT [[DelftSolutions:Internal]]

Tannaz

2024-07-04T11:44:17Z

Tannaz: Tannaz moved page DelftSolutions:Tannaz to DelftSolutions:Internal

#REDIRECT [[DelftSolutions:Internal]]

DelftSolutions:Internal

2024-07-04T11:44:17Z

Tannaz: Tannaz moved page DelftSolutions:Tannaz to DelftSolutions:Internal

Category:Tannaz

2024-07-04T11:43:59Z

Tannaz: Tannaz moved page Category:Tannaz to DelftSolutions:Tannaz

#REDIRECT [[DelftSolutions:Tannaz]]

DelftSolutions:Internal

2024-07-04T11:43:59Z

Tannaz: Tannaz moved page Category:Tannaz to DelftSolutions:Tannaz

User:Tannaz

2024-07-04T11:43:41Z

Tannaz: Tannaz moved page User:Tannaz to Category:Tannaz

#REDIRECT [[:Category:Tannaz]]

DelftSolutions:Internal

2024-07-04T11:43:40Z

Tannaz: Tannaz moved page User:Tannaz to Category:Tannaz

DelftSolutions:Internal

2024-07-04T11:42:15Z

Tannaz: How to create a cloud server on Hetzner

Talk:Internal

2024-06-20T11:33:44Z

Tannaz: Blanked the page

Talk:Internal

2024-06-20T11:01:04Z

Tannaz: Created page with "== Packets in the Network layer == Packets are small pieces of data that are sent over a network. Data sent over computer networks, such as the Internet, is divided into packets. These packets can take different paths to reach their destination and then recombined by the computer or device that receives them. The network layer is responsible for moving these packets from one place to another. Because data is broken into packets, many people can use the same network at th..."

== Packets in the Network layer ==
Packets are small pieces of data that are sent over a network. Data sent over computer networks, such as the Internet, is divided into packets. These packets can take different paths to reach their destination and then recombined by the computer or device that receives them. The network layer is responsible for moving these packets from one place to another. Because data is broken into packets, many people can use the same network at the same time for exchanging data.

=== IP Packets ===
IP packets are the fundamental units of data that are transmitted across IP (Internet Protocol) networks which is a network that uses the Internet Protocol (IP) to send and receive data, including the Internet, they actually route the packet to its destination. Packets are sometimes defined by the protocol they are using. They contain important information about where a packet is from (its source IP address), where it is going (destination IP address), how large the packet is, and other information about the data.
==== How Do IP Packet Work ====
When we send data over the internet, it is broken down into smaller pieces called packets. Each packet is sent independently and may travel different paths to reach the destination. At the destination, the packets are reassembled to form the original data. IP Packets have some components: the header and the payload. A packet header is a "label" of sorts, which provides information about the packet’s contents, origin, and destination.

In a network, data is sent in small pieces called packets. Each packet has different headers added by various protocols, which are rules for formatting and sending data. These headers help ensure the data gets where it needs to go and is understood by any computer on the network.
 

'''Packets often have more than one header. Each header is used by a different part of the networking process.'''
 

Actually protocols are standardized ways to format and handle data so that any computer can understand it. There is some common protocols: 
* Transmission Control Protocol (TCP): Ensures data is sent reliably and in order.
* Internet Protocol (IP): Routes the packet to its destination

==== IP Components ====
* Header

# Version: Indicates the IP version (IPv4 or IPv6).
# Header Length: Specifies the length of the header.
# Type of Service: Indicates the priority of the packet.
# Total Length: The combined length of the header and payload.
# Identification: Used to identify fragments of a larger packet.
# Flags: Control or identify fragments.
# Fragment Offset: Indicates where in the original packet this fragment belongs.
# Time to Live (TTL): Limits the packet's lifetime to prevent it from circulating indefinitely.
# Protocol: Indicates the protocol used in the data portion (e.g., TCP, UDP).
# Header Checksum: Used for error-checking the header.
# Source IP Address: The IP address of the sender.
# Destination IP Address: The IP address of the receiver.
# Options: Additional fields that provide extra features, optional.
 
* Payload
The payload is the actual data being transported, such as part of a file, an email, or a web page. This is the information the sender wants to deliver to the receiver.

 
When you send an email, the data is broken into packets. Each packet gets:

* IP Header: Contains the sender's and receiver's IP addresses, helping the packet find its destination.
* TCP Header: Ensures the packet arrives correctly

By adding these headers, protocols make sure that data can travel across the internet smoothly and be reassembled correctly at the destination.

=== Conclusion===
IP packets are essential for transmitting data across networks. They break down data into manageable chunks, route them efficiently, and ensure they arrive intact and in the correct order. This process is fundamental to the functioning of the Internet and modern digital communications. In summary, an IP network is any network that uses the Internet Protocol to manage data transmission, making it possible for devices to communicate efficiently.