How to properly remove a node from cluster Find your node then drain it to let k8s reschedule pods and avoid future schedule on this node: kubectl drain <node-name> --ignore-daemonsets --delete-local-data Then you’ll fine node.kubernetes.io/unschedulable=NoSchedule label on this node. Delete node from cluster. kubectl delete node <node-name> Then everything k8s related will be removed, and you’ll only see this left on node: t login: Fri Dec 6 05:25:27 2019 from 10.

Continue reading

Rook CRD

Rook is a Cloud Native Storage solution, it creates CRDs which in turn create their corresponding storage pods and resources. Install Rook CRD Install Operator via helm chart. This is the foundation of all fun. helm repo add rook-release https://charts.rook.io/release helm install --namespace rook-ceph rook-release/rook-ceph -n rook Note: Rook Operator and CRD cluster must be in the same namespace, because CRD will use helm created serviceaccount to create all resources.

Continue reading

Mariadb in a Galera Cluster Maintenance and Recovery¶ Introduction¶ This document covers how to perform system maintenance with the Mariadb database in active production, and how to recover from power outages, or network failure. Environment¶ SCM (Scyld Cloud Manager) currently leverages the Kolla OpenStack project which packages the OpenStack services into Docker containers. The mariadb database utilizes galera to run a database cluster on the three OpenStack controller systems. The cluster provides high availability as well as scalability.

Continue reading

Terraform Hints

Terraform now provides new functions, making itself more powerful as a configuration mgmt tool. Here are some memos: To make var value join with fixed string: "test-${each.key}" cidrsubnet From Terraform official documents, they have an example as following and explained in none humane language way. So I’ll try to interpret it from a Network Expert perspective. cidrsubnet(prefix, newbits, netnum) And an example below: > cidrsubnet("172.16.0.0/12", 4, 2) 172.18.0.0/16 > cidrsubnet("10.1.2.0/24", 4, 15) 10.

Continue reading

RedHat Certified System Engineer(EX300) is an advanced and extended exam from EX200. It tests what EX200 already covers and extend them in a more detailed and advanced level. Network Manager Besides what we already know about using nmcli to configure networks, RHCE would like to see if candidates would understand how nmcli and legacy script based config work. A table of Comparison of nm-settings and ifcfg-* Directives nmcli con mod ifcfg-* file Effect ipv4.

Continue reading

RedHat Certified System Administrator(EX200) is the 1st to acquire for most of Redhat advanced certificates(non-cloud, regular system admin path), Here are some hints and memo for learning and preparing for RHCSA. Link Files ln has hard link and soft link. ln newfile.txt /tmp/newfile-hlink2.txt will create a hard link of newfile.txt. hard link means full copy of the original file, the ln created hard link file will exist even if the original file got deleted.

Continue reading

All config and cmd in this blog has been verified and tested against Openshift 3.11 release Openshift is Redhat Container Platform, it mainly uses Kubernetes as its PaaS underlay and added more feature such as CICD, app store, etc. How to Install Similar as Kubespray, it uses a toolbox which has root access to all nodes and run ansible scripts to install and deploy everything. Few prerequisites before install:

Continue reading

Openstack Magnum

Magnum is the container cluster orchestration tool for Openstack, it uses Heat to deploy and monitor. The actual workflow would be: Python script to load cluster request –> Inject into Heat templates –> Start building VM –> run conditional actions in Shell –> build all nodes. Prerequisite Few setup need to be done before using Magnum: Node image needs to have property ‘os_distro’ set, fedora requires os_distro=fedora-atomic and coreos needs os_distro=coreos.

Continue reading

Openstack Octavia

All config and cmd in this blog has been verified and tested against Queens release Considering Neutron LBaaS has been replaced by Octavia and marked as depreciated since Queens, I think it’s time to write a brief blog about Octavia. LB is the key to many app services running on Openstack, and it’s critical for K8s environment as it’s the only ingress endpoint for a exposed service. Let’s firstly talk about the issues and weakness that current LBaaS has:

Continue reading

HyperFlex Hints

Upgrade and Mantainance Tool HyperFlex Cluster can be managed by using daemon agent on ESXi hosts directly, users may ssh into agents and issue stcli commands: stcli cluster start or stcli cluster stop can bring down/up entire ceph cluster. stcli node maintenanceMode --ip <hostIP> --mode enter and put a host into HX MM. Upgrade HX agents only: stcli cluster upgrade --components hxdp \ --location /tmp/storfs-packages-1.

Continue reading

Author's picture

LuLU

Love coding and new technologies

Cloud Solution Consultant

Canada