-
OCP Upgrade with Canary Rollout Strategy
Node upgrades are a critical aspect of maintaining a healthy OpenShift cluster. Whether it’s applying security patches, updating underlying dependencies, or simply scaling up resources, the process must be executed with precision to avoid disruptions to running workloads. OpenShift(Kubernetes) node upgrade methods often involve draining nodes, evacuating workloads, and performing the upgrade, leading to potential… Continue reading
-
OpenShift Global Pull Secret
In this post, we’ll walk you through how to securely fetch registry user credentials from HashiCorp Vault and apply them globally as a Pull Secret configuration in your OpenShift cluster. In today’s dynamic containerized environments, ensuring the security of your registry credentials is paramount. Exposing these credentials can lead to unauthorized access, data breaches, and… Continue reading
-
Vault Patch Method
With the release of HashiCorp Vault 1.9.0, managing secrets has become even more streamlined and efficient. One of the standout features? The ability to update only specified secrets, giving users precise control over their credentials within Vault. Imagine this: You want to update particular field(secret) in Vault location without touching others. Here, I shared a… Continue reading
-
check-certificate.sh
check-certificate.sh is a shell script which can be used to notify group of people over Slack before TLS certificates expire. Script can check tls certificates in two sources. tls:// and vault://. VAULT_ADDR, VAULT_TOKEN and SLACK_WEBHOOK must be set before the execution. check.txt You can find check-certificate.sh in my GH repo. Continue reading
-
walk.sh(HC Vault)
walk.sh script helps you print all secrets inside the KV type secret engine. You can extend the script to search for specific content. For example checking expire date of TLS certificates in your Vault before they expire. You can find walk.sh in my GH repo. You can find how-to in the repo. Continue reading
-
Slack Notification and Opsgenie Alerting in Jenkins
Alerting/Notification is an essential part of Tech operations to make sure business continuity and mitigating the financial loss. Nevertheless, self-healing platforms(e.g Kubernetes) on the market, there are some cases that human intervention is indispensable. In the mean time, proper alerting/notification system are crucial. E-mail notification quite outdated, and can be easily ignored by Operators due… Continue reading
-
nginx-ingress Websocket Connection
I had an issue with minio web console that, it stuck at “Loading” message while viewing “Object Browser”. Another issue with portainer that shell console closed immediately. They all sound to me due to the same exact issue. I figured out with help of Browser developer tools, that it logs very same message in the console. Solution: Adding following annotations to… Continue reading
-
Issuing Certificate with cert-manager
cert-manager is a X.509 certificate controller that allows you issue certificate from variety of certificate issuers and renew certificates automatically before they expire in your containerized environment. You can find more info about installation cert-manager Configuration Once cert-manager is installed first thing to be configured is Issuer or ClusterIssuer. These two resources represent Certificate Authority able to sign certificates in… Continue reading
-
HashiCorp Vault Disk Inode Is Full
Recently, I had very interesting issue with Vault that, it stopped functioning two hours after integration with Redfish Bare-Metal Host Monitoring which uses Approle to get iLO user and credentials. Vault audit logs shows that Approle for redfish monitoring requested heavy in a second. It might be still normal for around 80 Physical servers polling, but something… Continue reading
-
CheckMK Programmatic Hostdowntime
Sometimes you may have situation that you have to set/remove check_mk hostdowntime through multiple hosts due to planned maintenance to avoid false notifications and keep “unplanned maintenance” times correct for SLA reports. The GitHub repo contains both Ansible playbooks and shell scripts to set/remove/show host downtimes programmatically in check_mk. Both Ansible playbooks and shell scripts in GH repo are tested on check_mk version 2.0.0p31… Continue reading