October 7th, 2020
How are stateful workloads handled ?
How are distributed pods able to access the data all time ?
How is accessible data made consistent ?
And at times of failure , How is data retained ?
These are some of the questions that intrigues most of us while running our application on an High Available (HA) Openshift cluster .Let us find the answers
Firstly the pod workloads are handled by distributing it across the cluster to make the application highly available, it can be done by:
Using replica sets and Anti-Affinity rules
- Replica sets gives us the pod count, increased number of pods would ensure that specified number of pods are available in an healthy state at any point of time.
- Pod workload is handled by scheduling these pods on different computable nodes of the cluster and also making sure that no two similar app pods are on same node
- For this purpose, combination of nodeAffinity and podAntiAffinity rules are set so that the application pods are distributed evenly.
Secondly, data consistency, accessibility and retainability is achieved by provisioning NFS storage that uses persistent volumes.
- Here, NFS acts as persistent file storage device and we can make this storage instance available to cluster using static provisioning which requires creation of persistent volumes manually
- Hence, NFS acts as persistent file storage device and creation of NFS directory acts as a shared volume
Below is the procedure for setting up :
1. Creating NFS-server EC2 Instance
- Launch an Amazon Linux 2 AMI EC2 Instance (supports centos, rhel and fedora)
- Configure Instance details with Openshift cluster VPC , associate related public subnet in the same zone , proceed with default settings.
- configure security groups , allow NFS port 2049 .
- proceed for launching the instance .
- Now access the AWS terminal through putty and proceed for NFS- server installation
2. Installing the NFS-server
- Install these packages on the CentOS server with yum
- Create the directory that will be shared by NFS:
- Change the permissions of the folder as follows:
- Start the services and enable them to be started at boot time:
- Share the NFS directory over the network as follows:
- Finally , start the NFS service
yum install nfs-utils
chmod -R 755 /var/nfsshare
chown nfsnobody:nfsnobody /var/nfsshare
systemctl enable rpcbind
systemctl enable nfs-server
systemctl enable nfs-lock
systemctl enable nfs-idmap
systemctl start rpcbind
systemctl start nfs-server
systemctl start nfs-lock
systemctl start nfs-idmap
/var/nfsshare 192.168.100.2 (rw,sync,no_root_squash,no_all_squash) ##192.168.100.2 is the NFS client IP
## Validate with below command
showmount -e 192.168.100.1 ## gives exports list.
systemctl restart nfs-server
This completes setting up of NFS-server
3. At NFS-Client End
- Connect to worker nodes of openshift cluster which are NFS-Clients .
- create the NFS directory mount points:
- we will mount the /var/nfsshare directory:
- check if its mounted correctly :
## NFS Client installation inside worker nodes
yum install nfs-utils # need to install for non-coreOS
mkdir -p /mnt/nfs/var/nfsshare
mount -t nfs 192.168.100.1 :/var/nfsshare /mnt/nfs/var/nfsshare/ #192.168.100.1 is the NFS-server IP
4. Deploying Storage Class
- Below is the sample storage class used , one can edit storage class name and the provisioner name accordingly.
- Once updated the class.yml file we can execute the file using oc create
. archiveOnDelete: "false"
# create storage class file
oc create -f class.yml
#check if storage class got created
oc get storageclass
5. Creating Persistent Volume (PV)and Persistent Volume Claims (PVC)
- PV contains file storage details like NFS server IP and NFS shared path, thus it gets pointed to file storage device , that is to the NFS server. /b
- Using a reclaim policy of “retain”, a persistent volume will not be deleted or erased. Hence the relationship between pod and storage can be re-established
## create pv
server: 192.168.100.1 ## nfs server IP
path: /var/nfsshare ## nfs server shared path
## creat pvc
namespace: < project name >
volumeName: pv-nfs ## should be same as pv name
## to create above scripts
oc create -f pv.yml
oc create -f pvc.yml
6. Creating Pods to use Persistent Volume Claims
- Pods utilize this shared volume using persistent volume claim(PVC)
- name: deploy-nfs1
- name: pv-nfs
- key: kubernetes.io/hostname
- < worker1 hostname >
- < worker2 hostname >
- < worker3 hostname >
- key: app
- name: pv-nfs
# create the pod using deployment.yml file
oc create -f deployment.yml
# check for pods status
oc get pods -n < project name >
7. NFS Validation
- NFS-Server end, create a sample file and check if getting replicated to nfs client and in pod as well.
- NFS-Clients end, create a sample file and check if getting replicated to nfs server and in pod as well.
- At Pod level, create a file in the container mount path i.e /mydata , it would get reflected in NFS-server and in NFS-client machines.
we can now make our data persistent , accessible and retainable through NFS by means of static provisioning that requires manual creation of persistent volumes.
Whats ahead ?
Can we claim the persistent volume without actually creating it ?
Yes , without actually creating a persistent volume we can allocate storage volume just by creating persistent volume claim , this reduces time in bringing up the application.
In next post we will see how a dynamically provisioned NFS storage is set up .
DevOps Engineer| hands-on with Cloud Platform Infrastructure-AWS| Redhat Openshift Certified