Elucidating How to Maintain Data Consistency, Accessibility and Retainability while using High Available (HA) Openshift clusters
October 7th, 2020
How are stateful workloads handled ?
How are distributed pods able to access the data all time ?
How is accessible data made consistent ?
And at times of failure , How is data retained ?
These are some of the questions that intrigues most of us while running our application on an High Available (HA) Openshift cluster .Let us find the answers
Firstly the pod workloads are handled by distributing it across the cluster to make the application highly available, it can be done by:
Using replica sets and Anti-Affinity rules
- Replica sets gives us the pod count, increased number of pods would ensure that specified number of pods are available in an healthy state at any point of time.
- Pod workload is handled by scheduling these pods on different computable nodes of the cluster and also making sure that no two similar app pods are on same node
- For this purpose, combination of nodeAffinity and podAntiAffinity rules are set so that the application pods are distributed evenly.
Secondly, data consistency, accessibility and retainability is achieved by provisioning NFS storage that uses persistent volumes.
- Here, NFS acts as persistent file storage device and we can make this storage instance available to cluster using static provisioning which requires creation of persistent volumes manually
- Hence, NFS acts as persistent file storage device and creation of NFS directory acts as a shared volume
Below is the procedure for setting up :
1. Creating NFS-server EC2 Instance
- Launch an Amazon Linux 2 AMI EC2 Instance (supports centos, rhel and fedora)
- Configure Instance details with Openshift cluster VPC , associate related public subnet in the same zone , proceed with default settings.
- configure security groups , allow NFS port 2049 .
- proceed for launching the instance .
- Now access the AWS terminal through putty and proceed for NFS- server installation
2. Installing the NFS-server
- Install these packages on the CentOS server with yum
- Create the directory that will be shared by NFS:
- Change the permissions of the folder as follows:
- Start the services and enable them to be started at boot time:
- Share the NFS directory over the network as follows:
- Finally , start the NFS service
yum install nfs-utils
chmod -R 755 /var/nfsshare
chown nfsnobody:nfsnobody /var/nfsshare
systemctl enable rpcbind
systemctl enable nfs-server
systemctl enable nfs-lock
systemctl enable nfs-idmap
systemctl start rpcbind
systemctl start nfs-server
systemctl start nfs-lock
systemctl start nfs-idmap
/var/nfsshare 192.168.100.2 (rw,sync,no_root_squash,no_all_squash) ##192.168.100.2 is the NFS client IP
## Validate with below command
showmount -e 192.168.100.1 ## gives exports list.
systemctl restart nfs-server
This completes setting up of NFS-server
3. At NFS-Client End
- Connect to worker nodes of openshift cluster which are NFS-Clients .
- create the NFS directory mount points:
- we will mount the /var/nfsshare directory:
- check if its mounted correctly :
## NFS Client installation inside worker nodes
yum install nfs-utils # need to install for non-coreOS
mkdir -p /mnt/nfs/var/nfsshare
mount -t nfs 192.168.100.1 :/var/nfsshare /mnt/nfs/var/nfsshare/ #192.168.100.1 is the NFS-server IP
4. Deploying Storage Class
- Below is the sample storage class used , one can edit storage class name and the provisioner name accordingly.
- Once updated the class.yml file we can execute the file using oc create
. archiveOnDelete: "false"
# create storage class file
oc create -f class.yml
#check if storage class got created
oc get storageclass
5. Creating Persistent Volume (PV)and Persistent Volume Claims (PVC)
- PV contains file storage details like NFS server IP and NFS shared path, thus it gets pointed to file storage device , that is to the NFS server. /b
- Using a reclaim policy of “retain”, a persistent volume will not be deleted or erased. Hence the relationship between pod and storage can be re-established
## create pv
server: 192.168.100.1 ## nfs server IP
path: /var/nfsshare ## nfs server shared path
## creat pvc
namespace: < project name >
volumeName: pv-nfs ## should be same as pv name
## to create above scripts
oc create -f pv.yml
oc create -f pvc.yml
6. Creating Pods to use Persistent Volume Claims
- Pods utilize this shared volume using persistent volume claim(PVC)
- name: deploy-nfs1
- name: pv-nfs
- key: kubernetes.io/hostname
- < worker1 hostname >
- < worker2 hostname >
- < worker3 hostname >
- key: app
- name: pv-nfs
# create the pod using deployment.yml file
oc create -f deployment.yml
# check for pods status
oc get pods -n < project name >
7. NFS Validation
- NFS-Server end, create a sample file and check if getting replicated to nfs client and in pod as well.
- NFS-Clients end, create a sample file and check if getting replicated to nfs server and in pod as well.
- At Pod level, create a file in the container mount path i.e /mydata , it would get reflected in NFS-server and in NFS-client machines.
we can now make our data persistent , accessible and retainable through NFS by means of static provisioning that requires manual creation of persistent volumes.
Whats ahead ?
Can we claim the persistent volume without actually creating it ?
Yes , without actually creating a persistent volume we can allocate storage volume just by creating persistent volume claim , this reduces time in bringing up the application.
In next post we will see how a dynamically provisioned NFS storage is set up .
DevOps Engineer| hands-on with Cloud Platform Infrastructure-AWS| Redhat Openshift Certified
How EDI is Shape Shifting Your Legacy Systems
Communication and data transfers are no longer elective activities for a business in both B2B and B2C models. And Electronic Data Interchange (EDI) plays a significant role in establishing viable communications. Organizations that stay up-to-date in this regard achieve greater success.
EDI has been around for a while, and it helps in the standardization of the data transfer process. But the long-standing technology is currently under scrutiny. It is because organizations have become much more complex, and their enterprise-level requirements are growing on a global scale.
Yes, legacy EDI systems still exist, but they can pose new challenges. Modernization of EDI protocols and processes is the only solution. This process of modernization of legacy systems is particularly essential if one of the business partners is still stuck with older technologies. Emerging solutions are the order of the day to improve their profits.
Diverse legacy EDI systems become fragmented and unwieldy. For example, Value-added networks (VAN) get disjointed and require multiple tools. Otherwise, it is not possible to ensure file transfer and mapping or back-end integration. Such systems can also stretch the wits of the trouble-shooters.
Workflows get slowdown as different tools are essential to process orders. Communicating with trade partners and system upgrades also become challenging. But modernizing EDI improves end-to-end workflows. It also enhances supply chain visibility and facilitates faster upgrades and system repairs.
Legacy systems can challenge the HR resources of any company. The staff has to undergo training for diverse tools usage. It makes the process of building teams, and expertise takes time and cost money too. Not all employees will be on board and thrilled by the prospects of reading lengthy manuals, either.
Modern EDI systems have convenient UI/UX and friendly drag-and-drop features. They eliminate coding needs and provide a single, seamless front-end for quick mastery. The visual interfaces appeal to the users, and they even show a willingness to do remote work.
Asynchronous file transfer and batch processing were the banes of older EDI systems. These processes were slow, and scheduling demands stressed out the managers. But now, cloud computing and mobile phone access is a reality.
Data collection, storage, and back-end processing are faster. They happen in real-time, and syncing features resolve data redundancies and replication issues. Cost control becomes efficient and overall productivity and profits of the organization increase.
In the future, companies that adapt to modernity will thrive. They will have an innovative edge over their competitors on a global scale. Technologies and tools have also become more reliable and refined. Web interfaces, robust APIs, mobile data collection, and superior security are assuring.
Data decentralization through Blockchain technology enhances transparency. The modern EDI tools are faster, convenient, and more reliable.
Older IT systems and legacy EDI tools have multiple security flaws. They are vulnerable and require constant upgrades and updates. This requirement increases maintenance costs and disrupts the workflows. But modern EDI VANs ensure data security, compliance, and optimized supply chains. They reduce the risks of data storage and file transfer and address privacy issues also.
Modern EDI solutions are not in conflict with APIs. The developers are endorsing their simultaneous usage. Legacy system problems like rigidity and inflexibility can get dealt with API-first architectures. Cutting-edge interfaces like JSON, REST, and OData create feature-rich apps. Besides, customized, code-free APIs enhance workflow integration, data access, and publication. Developers can provide secure, real-time access, even to legacy databases, Excel sheets, etc.
Integration with legacy EDI requires considerable investment. The specialists have to set-up expensive servers, cloud interfaces, web, and mobile apps. Modern solutions reduce such complexity through container technologies. The project managers can design scalable solutions in advance, and even implement them quickly and accurately with superior software development tools.
Legacy desktop applications rely on version control systems and package repositories. It facilitated data maintenance and collaboration to a limited extent, but modern EDI solutions create an eco-system using APIs, web, and mobile interfaces. So, collaboration becomes more rapid and reliable.
Traditional IT infrastructure gets limited by platform dependencies. It extends from hardware to operating system requirements. Thankfully, modern EDI deployment and integration are platforms- agnostic. You can set up the system on UNIX and Windows, or run it as a cloud-based application.
This flexibility accelerates end-to-end workflows without compromising security and privacy. Moreover, the modern system is not only interoperable but also easy to maintain and upgrade. It also addresses the organization’s business needs without a breakdown in supply chain processes.
To conclude, modern businesses operate in a dynamic environment. They are under pressure to compete and resolve complex problems very quickly. Mere automation of business processes and data communications is not sufficient anymore.
Modernizing EDI is not easy for business partners in e-commerce. They have to deal with legacy systems and spend on new technology. Also, integration with existing IT infrastructure is challenging. But the emerging solutions are too enticing, and safety in familiarity is not an option anymore.
CloudGen’s SmartEDI is one such solution, which can be a perfect fit into any of the B2B transactions. Easy setup and implementation with accelerated onboarding process is our promise to the struggling new companies, to head out successfully in taking off the EDI systems. Contact us for a live demo, which helps you to understand more.
Orders, invoices, catalogues, instructions, sales reports, quotations, and more – the list of documents required in logistics management is endless. Logistics and supply chain is heavily reliant on documents for transactions and communications. But this increased paperwork translates to the increased expenditure of time, money, and resources. And don’t forget the risk of errors.
Electronic data exchange (EDI) is a computer-to-computer exchange of business documents, which allows you to eliminate cumbersome paperwork, reduce costs, and improve efficiency.
In this article, you’ll learn about the key benefits of EDI in logistics and supply chain.
Let’s jump right in.
First things first, EDI helps you save costs. As discussed, logistics management comprises copious amounts of paperwork, including printing, storage, reproduction, postage, and retrieval. All this can add significantly to the overall logistics expenses. Utilizing an EDI system enables you to lower your transaction costs by 35-45%.
Besides, manual administration of documents is susceptible to illegible faxes, incorrectly taken orders, or lost orders, which can add up to the losses. EDI eliminates human involvement, thereby alleviating the risk of losses.
Lastly, manual documentation can eat up your valuable resources. For instance, you’ll need to have an employee present to handle the tasks, along with machinery, to print, send, and receive paper documents. EDI automates the entire process and allows you to make better utilization of your resources.
Improved Speed and Accuracy
Electronic data interchange can accelerate your supply chain procedures by up to 60%. For example, it can speed up exchange transactions to a few minutes, instead of waiting for days in case of postal service. Apart from speed, EDI also enhances data quality by delivering a minimum of 30-40% reduction in transaction errors. It excludes mistakes emerging from illegible handwriting, keying/re-keying errors, or lost mail/faxes.
Using EDI can also help you reduce the order-to-cash cycle time by 20-30%, thus, improving business partner relationships and transactions. In addition, using EDI enables you to handle a significantly larger task load with better accuracy.
Improved Business Efficiency
EDI automates paper-based tasks, which allows you and your staff to focus on more productive, higher-value activities. Besides, it helps you and your transport management provider to spend more time on hand-getting shipments where they need to go, instead of spending hours juggling through paper documents and email communications.
Another benefit of EDI in supply chain management is the quick and accurate processing of business documents, which results in fewer stock-outs, cancelled orders, and re-working of orders. It ensures you’re able to send business-critical data on time and track it in real-time.
EDI further allows you to improve cash flow and reduce order-to-cash cycles, along with shortening order processing and delivery times to reduce your inventory levels.
Streamlined Communication Between Partners
Communication plays a key role in the success of logistics operations. However, paper-based documentation impedes communication between trading partners. In addition to streamlining transactions, EDI also advances all types of trading partner interactions.
This results in improved partner-customer relationships and more sales. For example, EDI allows you to shorten your sales cycle. So, you can take and confirm orders more rapidly, giving your customers confidence in your ability to deliver remarkable service.
Peak Inventory Management
Maintaining optimal inventory levels is one of the hurdles most logistics managers face. Not having the right inventory levels means you won’t be able to meet your customer demands, which will lead to losses and a degraded reputation.
EDI helps you maintain ideal inventory stock levels for the future. It provides you access to real-time updates and latest market trends, which you can use to streamline proper resource allocation and manage shortage or excess inventory levels.
Promotes Strategic Planning
One of the major hurdles logistics companies and managers encounter is the lack of transparency and visibility into the status of transportation transactions. EDI resolves this challenge by offering real-time visibility into transportation transactions, which, in turn, promotes faster decision-making and enhanced responsiveness to changing market demands and customer preferences.
EDI also conduces to strategic business planning in the following ways:
- Cuts down the lead times for product improvements and new product delivery
- Enhances corporate social responsibility and sustainability by replacing paper-based procedures with electronic methods. This helps you save money and carbon dioxide emissions.
- Enables you to enter new markets and territories by providing a common business language for facilitating business partners across the globe.
We live in the era of digitization, where businesses need to evolve and amp-up their operations. Electronic data exchange allows you to gain more information on your fingertips in the quickest possible time. You can trace orders and see their status in real-time.
EDI also helps you manage your inventory levels and enables you to keep up with the fluctuating market demand. On the whole, EDI promotes strategic planning and gives your businesses an edge over others.
CloudGen has the right smart solution called SmartEDI, which can perfectly fit into your Logistic needs and make the business happen in the most acceptable desired format. Our team of experts never fail to satisfy customer expectation and always maintain maximum delivery excellence.
Electronic Data Interchange was not born yesterday. It has been around for decades, and in spite of several developments and revolutions, EDI has remained a pillar in the realm of B2B communications and collaborations.
And as new technologies develop, it is critical to reevaluate the role of electronic data interchange in the supply chain.
Let’s take a look at how EDI is changing the world especially B2B landscape, along with some EDI trends for 2020 and beyond.
As per the International Trade Administration, less than one percent of the 30 million US companies export goods, and 58% export to only one country. These stats indicate that companies should focus on more international collaborations and compete in the global market. And for the suppliers to adequately meet the changing demands of the customers, automating the processes electronically is essential.
Implementing EDI automates the supply chain process, which allows buyers to pay suppliers easily and quickly without any potential supply chain disruptions. In the case of suppliers, the supply chain automation offers easier compliance with tons of buyer standards, which significantly alleviates the risk of penalties, like chargebacks.
Therefore, more and more companies are likely to enter into global-scale collaborations and expand their presence across borders.
The Rise of the Cloud
Firstly, the cloud is not a new or emerging concept anymore, but still, there’s a lot of progress to be made to systemize it as a global business practice. Currently, 64% of distribution centres have a warehouse management system, but only 8% of them are cloud-based. In a few years down the line, more organizations will transition to a cloud-based infrastructure.
Besides, the cloud allows you to scale seamlessly. For example, let’s say you’re a sports product manufacturer, and you’ve recently moved from district sales to a broker model. Now, you’re targeting large clients like multinational sports retail outlets, and your brokers are being incentivized to close big clients. Your sales and revenue surged, but so did the demand of the consumers.
In order to distribute the products to thousands of distribution centres and millions of customers, you and the retailer will require specific EDI compliance capabilities. By utilizing the power of the cloud, you would be able to scale your resources and meet fluctuating demands without batting an eye.
Further, this seamless scalability has made the cloud an essential aspect of warehouse and logistics companies around the globe.
The Revival of In-House Competencies
Until recently, most companies outsourced their EDI initiatives. As of today, several businesses have shifted to in-house EDI processing. It helps save costs and power, along with increasing the level of security and customer experience.
But on the whole, the expense of in-house shifting could still be quite expensive. Therefore, this shift might take some time, starting with large-scale companies and enterprises. The trends, however, suggest that most companies will sooner or later embrace in-house EDI integration.
Electronic Data Interchange has enabled improved data capabilities for proactive management. When organizations operate together, the chances of losing forward momentum are high.
As more data enters a business, decision-makers improve their forecasting capabilities, enterprise resource planning, and visibility. It, therefore, becomes a predictable way to manage something unpredictable.
For instance, the rise of eCommerce has made the demand of products increasingly predictable. But with EDI, organizations can better manage how a product will reach its end customers.
A Shift Towards Omnichannel Capabilities
The major suppliers have their inventory sitting in distribution centres across various locations and divisions. To allow customers to purchase products through a range of connected interfaces, companies must manage their inventory across an array of global distribution points.
Omnichannel selling is one o the ways businesses can use to expand their distribution. More often than not, it is impossible for supplies to hold the inventory of all their buyers 100% of the time. By leveraging an omnichannel strategy, organizations can expand their number of stock-keeping units and cater to the various tastes of their customers.
Let’s take, for example, dropship retailers who must provide the suppliers with continuously-branded packing slips. Besides, they should also continuously update their inventory to make whatever is in the stock at the moment available to the retailer.
EDI allows suppliers to shift smoothly towards omnichannel capabilities by providing actionable insights. This helps in protecting the buyer’s brand, ensuring customer satisfaction, and promoting B2B integration. Moreover, the increased use of EDI has encouraged organizations to embrace omnichannel selling to create an endless aisle for their customers.
It’s too soon to say what the future holds for EDI. The rise of the internet, IPaaS, and technologies like blockchain, on the one hand, suggests that EDI will soon equip more power by merging itself with smart technologies. But on the other hand; EDI itself is still the reigning choice for businesses as it gets the job done, and everyone else is using it. So, in conclusion, it’s worth the wait to watch what EDI will offer in the future.
CloudGen’s SmartEDI product enhances the efficiency of EDI to the next new heights which completely focuses on implementing EDI with ease at every stage. Our product provides a simple solution to overcome the setup complexities with reduced cost and time investments assuring the productivity rise.
Breaking it all the way down for everyone, what is IoT and why is it essential, we here at CloudGen are investing heavily in IoT and think you need to be too! Here is why.