Elucidating How to Maintain Data Consistency, Accessibility and Retainability while using High Available (HA) Openshift clusters

October 7th, 2020

How are stateful workloads handled ?

How are distributed pods able to access the data all time ?

How is accessible data made consistent ?

And at times of failure , How is data retained ?

These are some of the questions that intrigues most of us while running our application on an High Available (HA) Openshift cluster .Let us find the answers

Firstly the pod workloads are handled by distributing it across the cluster to make the application highly available, it can be done by:

Using replica sets and Anti-Affinity rules

nfs1

  • Replica sets gives us the pod count, increased number of pods would ensure that specified number of pods are available in an healthy state at any point of time.
  • Pod workload is handled by scheduling these pods on different computable nodes of the cluster and also making sure that no two similar app pods are on same node
  • For this purpose, combination of nodeAffinity and podAntiAffinity rules are set so that the application pods are distributed evenly.

Secondly, data consistency, accessibility and retainability is achieved by provisioning NFS storage that uses persistent volumes.

  • Here, NFS acts as persistent file storage device and we can make this storage instance available to cluster using static provisioning which requires creation of persistent volumes manually
  • Hence, NFS acts as persistent file storage device and creation of NFS directory acts as a shared volume

2

Below is the procedure for setting up :

1. Creating NFS-server EC2 Instance

  • Launch an Amazon Linux 2 AMI EC2 Instance (supports centos, rhel and fedora)
  • Configure Instance details with Openshift cluster VPC , associate related public subnet in the same zone , proceed with default settings.
  • configure security groups , allow NFS port 2049 .
  • proceed for launching the instance .
  • Now access the AWS terminal through putty and proceed for NFS- server installation

2. Installing the NFS-server

  • Install these packages on the CentOS server with yum
  • yum install nfs-utils

  • Create the directory that will be shared by NFS:
  • mkdir /var/nfsshare

  • Change the permissions of the folder as follows:
  • chmod -R 755 /var/nfsshare
    chown nfsnobody:nfsnobody /var/nfsshare

  • Start the services and enable them to be started at boot time:
  • systemctl enable rpcbind
    systemctl enable nfs-server
    systemctl enable nfs-lock
    systemctl enable nfs-idmap
    systemctl start rpcbind
    systemctl start nfs-server
    systemctl start nfs-lock
    systemctl start nfs-idmap

  • Share the NFS directory over the network as follows:
  • vim /etc/exports
    /var/nfsshare 192.168.100.2 (rw,sync,no_root_squash,no_all_squash) ##192.168.100.2 is the NFS client IP
    ## Validate with below command
    showmount -e 192.168.100.1 ## gives exports list.

  • Finally , start the NFS service
  • systemctl restart nfs-server

This completes setting up of NFS-server

3. At NFS-Client End

  • Connect to worker nodes of openshift cluster which are NFS-Clients .
  • ## NFS Client installation inside worker nodes
    yum install nfs-utils # need to install for non-coreOS

  • create the NFS directory mount points:
  • mkdir -p /mnt/nfs/var/nfsshare

  • we will mount the /var/nfsshare directory:
  • mount -t nfs 192.168.100.1 :/var/nfsshare /mnt/nfs/var/nfsshare/ #192.168.100.1 is the NFS-server IP

  • check if its mounted correctly :
  • df -h

4. Deploying Storage Class

  • Below is the sample storage class used , one can edit storage class name and the provisioner name accordingly.
  • #vim class.yml
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
    name: nfs-storage
    annotations:
    storageclass.kubernetes.io/is-default-class: "false"
    provisioner: example.com/nfs
    parameters:
    . archiveOnDelete: "false"

  • Once updated the class.yml file we can execute the file using oc create
  • # create storage class file
    oc create -f class.yml
    #check if storage class got created
    oc get storageclass

5. Creating Persistent Volume (PV)and Persistent Volume Claims (PVC)

  • PV contains file storage details like NFS server IP and NFS shared path, thus it gets pointed to file storage device , that is to the NFS server. /b
    • Using a reclaim policy of “retain”, a persistent volume will not be deleted or erased. Hence the relationship between pod and storage can be re-established

    ## create pv
    kind: PersistentVolume
    apiVersion: v1
    metadata:
    name: pv-nfs
    spec:
    capacity:
    storage: 1Gi
    accessModes:
    - ReadWriteMany
    persistentVolumeReclaimPolicy: Retain
    storageClassName: nfs-storage
    nfs:
    server: 192.168.100.1 ## nfs server IP
    path: /var/nfsshare ## nfs server shared path
    ## creat pvc
    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
    name: pvc-nfs
    namespace: < project name >
    spec:
    accessModes:
    - ReadWriteMany
    resources:
    requests:
    storage: 20Mi
    volumeName: pv-nfs ## should be same as pv name
    storageClassName: nfs-storage

    ## to create above scripts
    oc create -f pv.yml
    oc create -f pvc.yml

6. Creating Pods to use Persistent Volume Claims

  • Pods utilize this shared volume using persistent volume claim(PVC)
  • apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: deploy-nfs
    labels:
    app: deploy-nfs
    spec:
    replicas: 2
    selector:
    matchLabels:
    app: deploy-nfs
    template:
    metadata:
    labels:
    app: deploy-nfs
    spec:
    containers:
    - name: deploy-nfs1
    image: busybox
    command:
    - sleep
    - "3600"
    resources:
    requests:
    cpu: "0.01"
    volumeMounts:
    - name: pv-nfs
    mountPath: /mydata
    affinity:
    nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    nodeSelectorTerms:
    - matchExpressions:
    - key: kubernetes.io/hostname
    operator: In
    values:
    - < worker1 hostname >
    - < worker2 hostname >
    - < worker3 hostname >
    podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
    matchExpressions:
    - key: app
    operator: In
    values:
    - deploy-nfs
    topologyKey: "kubernetes.io/hostname"
    volumes:
    - name: pv-nfs
    persistentVolumeClaim:
    claimName: pvc-nfs

    # create the pod using deployment.yml file
    oc create -f deployment.yml
    # check for pods status
    oc get pods -n < project name >

7. NFS Validation

  1. NFS-Server end, create a sample file and check if getting replicated to nfs client and in pod as well.
  2. NFS-Clients end, create a sample file and check if getting replicated to nfs server and in pod as well.
  3. At Pod level, create a file in the container mount path i.e /mydata , it would get reflected in NFS-server and in NFS-client machines.

Takeaways

we can now make our data persistent , accessible and retainable through NFS by means of static provisioning that requires manual creation of persistent volumes.

Whats ahead ?

Can we claim the persistent volume without actually creating it ?

Yes , without actually creating a persistent volume we can allocate storage volume just by creating persistent volume claim , this reduces time in bringing up the application.

In next post we will see how a dynamically provisioned NFS storage is set up .

WRITTEN BY

Saicharan Adavelli

DevOps Engineer| hands-on with Cloud Platform Infrastructure-AWS| Redhat Openshift Certified

Traditionalizing the Modern Approach  How EDI is Shape Shifting Your Legacy Systems

Traditionalizing the Modern Approach

How EDI is Shape Shifting Your Legacy Systems

Communication and data transfers are no longer elective activities for a business in both B2B and B2C models. And Electronic Data Interchange (EDI) plays a significant role in establishing viable communications. Organizations that stay up-to-date in this regard achieve greater success.

EDI has been around for a while, and it helps in the standardization of the data transfer process. But the long-standing technology is currently under scrutiny. It is because organizations have become much more complex, and their enterprise-level requirements are growing on a global scale.

Yes, legacy EDI systems still exist, but they can pose new challenges. Modernization of EDI protocols and processes is the only solution. This process of modernization of legacy systems is particularly essential if one of the business partners is still stuck with older technologies. Emerging solutions are the order of the day to improve their profits.

Future-oriented Solutions

Workflow Unification

Diverse legacy EDI systems become fragmented and unwieldy. For example, Value-added networks (VAN) get disjointed and require multiple tools. Otherwise, it is not possible to ensure file transfer and mapping or back-end integration. Such systems can also stretch the wits of the trouble-shooters.

Workflows get slowdown as different tools are essential to process orders. Communicating with trade partners and system upgrades also become challenging. But modernizing EDI improves end-to-end workflows. It also enhances supply chain visibility and facilitates faster upgrades and system repairs.

User-Friendliness

Legacy systems can challenge the HR resources of any company. The staff has to undergo training for diverse tools usage. It makes the process of building teams, and expertise takes time and cost money too. Not all employees will be on board and thrilled by the prospects of reading lengthy manuals, either.

Modern EDI systems have convenient UI/UX and friendly drag-and-drop features. They eliminate coding needs and provide a single, seamless front-end for quick mastery. The visual interfaces appeal to the users, and they even show a willingness to do remote work.

Faster Productivity 

Asynchronous file transfer and batch processing were the banes of older EDI systems. These processes were slow, and scheduling demands stressed out the managers. But now, cloud computing and mobile phone access is a reality. 

Data collection, storage, and back-end processing are faster. They happen in real-time, and syncing features resolve data redundancies and replication issues. Cost control becomes efficient and overall productivity and profits of the organization increase.

Innovative Edge  

In the future, companies that adapt to modernity will thrive. They will have an innovative edge over their competitors on a global scale. Technologies and tools have also become more reliable and refined. Web interfaces, robust APIs, mobile data collection, and superior security are assuring.

Data decentralization through Blockchain technology enhances transparency. The modern EDI tools are faster, convenient, and more reliable. 

Superior Security

Older IT systems and legacy EDI tools have multiple security flaws. They are vulnerable and require constant upgrades and updates. This requirement increases maintenance costs and disrupts the workflows. But modern EDI VANs ensure data security, compliance, and optimized supply chains. They reduce the risks of data storage and file transfer and address privacy issues also.

Complementing APIs 

Modern EDI solutions are not in conflict with APIs. The developers are endorsing their simultaneous usage. Legacy system problems like rigidity and inflexibility can get dealt with API-first architectures. Cutting-edge interfaces like JSON, REST, and OData create feature-rich apps. Besides, customized, code-free APIs enhance workflow integration, data access, and publication. Developers can provide secure, real-time access, even to legacy databases, Excel sheets, etc.

Scalability 

Integration with legacy EDI requires considerable investment. The specialists have to set-up expensive servers, cloud interfaces, web, and mobile apps. Modern solutions reduce such complexity through container technologies. The project managers can design scalable solutions in advance, and even implement them quickly and accurately with superior software development tools.

Collaboration Support 

Legacy desktop applications rely on version control systems and package repositories. It facilitated data maintenance and collaboration to a limited extent, but modern EDI solutions create an eco-system using APIs, web, and mobile interfaces. So, collaboration becomes more rapid and reliable.

Flexibility

Traditional IT infrastructure gets limited by platform dependencies. It extends from hardware to operating system requirements. Thankfully, modern EDI deployment and integration are platforms- agnostic. You can set up the system on UNIX and Windows, or run it as a cloud-based application. 

This flexibility accelerates end-to-end workflows without compromising security and privacy. Moreover, the modern system is not only interoperable but also easy to maintain and upgrade. It also addresses the organization’s business needs without a breakdown in supply chain processes.

Conclusion

To conclude, modern businesses operate in a dynamic environment. They are under pressure to compete and resolve complex problems very quickly. Mere automation of business processes and data communications is not sufficient anymore.

Modernizing EDI is not easy for business partners in e-commerce. They have to deal with legacy systems and spend on new technology. Also, integration with existing IT infrastructure is challenging. But the emerging solutions are too enticing, and safety in familiarity is not an option anymore.  

CloudGen’s SmartEDI is one such solution, which can be a perfect fit into any of the B2B transactions. Easy setup and implementation with accelerated onboarding process is our promise to the struggling new companies, to head out successfully in taking off the EDI systems. Contact us for a live demo, which helps you to understand more.

blog 1 advantages

The Advantages of EDI in Logistics and Supply Chain

Orders, invoices, catalogues, instructions, sales reports, quotations, and more – the list of documents required in logistics management is endless. Logistics and supply chain is heavily reliant on documents for transactions and communications. But this increased paperwork translates to the increased expenditure of time, money, and resources. And don’t forget the risk of errors.

Electronic data exchange (EDI) is a computer-to-computer exchange of business documents, which allows you to eliminate cumbersome paperwork, reduce costs, and improve efficiency.

In this article, you’ll learn about the key benefits of EDI in logistics and supply chain.

Let’s jump right in.

Cost Savings

First things first, EDI helps you save costs. As discussed, logistics management comprises copious amounts of paperwork, including printing, storage, reproduction, postage, and retrieval. All this can add significantly to the overall logistics expenses. Utilizing an EDI system enables you to lower your transaction costs by 35-45%.

Besides, manual administration of documents is susceptible to illegible faxes, incorrectly taken orders, or lost orders, which can add up to the losses. EDI eliminates human involvement, thereby alleviating the risk of losses.

Lastly, manual documentation can eat up your valuable resources. For instance, you’ll need to have an employee present to handle the tasks, along with machinery, to print, send, and receive paper documents. EDI automates the entire process and allows you to make better utilization of your resources.

Improved Speed and Accuracy

Electronic data interchange can accelerate your supply chain procedures by up to 60%. For example, it can speed up exchange transactions to a few minutes, instead of waiting for days in case of postal service. Apart from speed, EDI also enhances data quality by delivering a minimum of 30-40% reduction in transaction errors. It excludes mistakes emerging from illegible handwriting, keying/re-keying errors, or lost mail/faxes.

Using EDI can also help you reduce the order-to-cash cycle time by 20-30%, thus, improving business partner relationships and transactions. In addition, using EDI enables you to handle a significantly larger task load with better accuracy.

Improved Business Efficiency

EDI automates paper-based tasks, which allows you and your staff to focus on more productive, higher-value activities. Besides, it helps you and your transport management provider to spend more time on hand-getting shipments where they need to go, instead of spending hours juggling through paper documents and email communications.

Another benefit of EDI in supply chain management is the quick and accurate processing of business documents, which results in fewer stock-outs, cancelled orders, and re-working of orders. It ensures you’re able to send business-critical data on time and track it in real-time.

EDI further allows you to improve cash flow and reduce order-to-cash cycles, along with shortening order processing and delivery times to reduce your inventory levels.

Streamlined Communication Between Partners

Communication plays a key role in the success of logistics operations. However, paper-based documentation impedes communication between trading partners. In addition to streamlining transactions, EDI also advances all types of trading partner interactions.

This results in improved partner-customer relationships and more sales. For example, EDI allows you to shorten your sales cycle. So, you can take and confirm orders more rapidly, giving your customers confidence in your ability to deliver remarkable service.

Peak Inventory Management

Maintaining optimal inventory levels is one of the hurdles most logistics managers face. Not having the right inventory levels means you won’t be able to meet your customer demands, which will lead to losses and a degraded reputation.

EDI helps you maintain ideal inventory stock levels for the future. It provides you access to real-time updates and latest market trends, which you can use to streamline proper resource allocation and manage shortage or excess inventory levels.

Promotes Strategic Planning

One of the major hurdles logistics companies and managers encounter is the lack of transparency and visibility into the status of transportation transactions. EDI resolves this challenge by offering real-time visibility into transportation transactions, which, in turn, promotes faster decision-making and enhanced responsiveness to changing market demands and customer preferences.

EDI also conduces to strategic business planning in the following ways:

  • Cuts down the lead times for product improvements and new product delivery
  • Enhances corporate social responsibility and sustainability by replacing paper-based procedures with electronic methods. This helps you save money and carbon dioxide emissions.
  • Enables you to enter new markets and territories by providing a common business language for facilitating business partners across the globe.

Conclusion

We live in the era of digitization, where businesses need to evolve and amp-up their operations. Electronic data exchange allows you to gain more information on your fingertips in the quickest possible time. You can trace orders and see their status in real-time.

EDI also helps you manage your inventory levels and enables you to keep up with the fluctuating market demand. On the whole, EDI promotes strategic planning and gives your businesses an edge over others.

CloudGen has the right smart solution called SmartEDI, which can perfectly fit into your Logistic needs and make the business happen in the most acceptable desired format. Our team of experts never fail to satisfy customer expectation and always maintain maximum delivery excellence.

Watch Out for 2020 EDI Trends and How EDI Is Changing the World

How EDI is Changing the World

Electronic Data Interchange was not born yesterday. It has been around for decades, and in spite of several developments and revolutions, EDI has remained a pillar in the realm of B2B communications and collaborations.
And as new technologies develop, it is critical to reevaluate the role of electronic data interchange in the supply chain.
Let’s take a look at how EDI is changing the B2B landscape, along with some EDI trends for 2020 and beyond.

IoT

IoT – Internet of Things

Breaking it all the way down for everyone, what is IoT and why is it essential, we here at CloudGen are investing heavily in IoT and think you need to be too! Here is why.

CloudGen © 2019 All Rights Reserved