Scaling Containerized Applications on AWS with Amazon EKS

Overview

Serviceo offers an innovative software platform that leverages over a decade of industry experience with flexible business models that enables organizations to streamline their global IT field service delivery. Their platform provides On-Demand field services around the globe, meeting enterprise SLAs utilizing 600 forward stocking locations around the world and a network of thousands of vetted and highly qualified local technicians.

As Serviceo continues to grow, the company needs to be able to deploy, test, validate and promote components rapidly in their micro-service and component-based architecture. The software development team was experiencing significant struggles to work as a unified team. Code was being committed and deployed with broken library dependencies, resulting in manual fixes every release to ensure code was running correctly. The applications were disjointed and inconsistent, with very loosely coupled sets of software, services, and scripts. There was no robust deployment of the applications, and once deployed, there was often a need to intervene manually. The team wanted to have a Zero Downtime Deployment (ZDD) strategy to deploy a new version of services without interrupting the operation of the service.

Solution

Working together, Avahi and Serviceo development team decided that the best approach is:

Containerize the applications.
Deploy the most efficient Continuous Integration/Continuous Deployment (CI/CD) pipeline.
Restructure the Git software repositories into specific areas of concern (apps/services/dependencies).
Adopt Amazon EKS for automating deployment, scaling, and management of containerized applications.

Avahi cloud engineers planned to build a more modular, extensible system to support future feature enhancements and a more frequent release cycle. Kubernetes-based solution for runtime can do that while providing the scale to meet present-day and future demands on the system. The solution also had to be secure enough to meet regulatory requirements, such as the California Consumer Privacy Act (CCPA) and General Data Protection Regulation (GDPR). Avahi proposed to use Amazon EKS for Kubernetes instead of building the cluster from the ground up.

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed container orchestration service that provides templates, code samples, and best practices to help developers ramp up quickly. AWS manages the infrastructure for Kubernetes, freeing the team to focus on adding value to their application – not on operations. The benefits of using Amazon EKS for Serviceo include:

Efficient resource utilization

The fully managed EKS offers easy deployment and management of containerized applications. It enables efficient resource utilization that elastically provisions additional resources without the headache of managing the Kubernetes infrastructure.

Faster application development

Developers spent most of the time on bug-fixing. EKS reduces the debugging time while handling patching, auto-upgrades, and self-healing and simplifies the container orchestration. It saves a lot of time, and developers will focus on developing their apps while remaining more productive.

Security and compliance

Cybersecurity is one of the most important aspects of modern applications and businesses. EKS integrates with AWS IAM and offers users on-demand access to reduce threats and risks. EKS is also completely compliant with the standards and regulatory requirements such as System and Organization Controls (SOC), HIPAA, ISO, and PCI DSS.

Quicker development and integration

EKS supports auto-upgrades, monitoring, and scaling and helps minimize the infrastructure maintenance that leads to comparatively faster development and integration. It also supports provisioning additional compute resources in Serverless Kubernetes within seconds without worrying about managing the Kubernetes infrastructure.

Approach

As a first phase, Avahi deployed a development workflow based on the Atlassian suite of products (JIRA/BitBucket). The goal was to enable rapid iteration of the team’s code while ensuring comprehensive software testing, quality control, and integration. The workflow relied on properly defined environments – Development, Testing, Acceptance, Production (DTAP).

Adopting the streamlined development workflow and restructured code management improved the collaboration across the teams and software quality. The leadership got much-improved visibility on what developers were pushing into the branches and its effect on the overall software. Each repository would contain its tests, dependency tree, and Bitbucket pipeline YAML config. It enabled more autonomy in development while retaining efficient control over the cross-platform dependencies and testing.

Avahi deployed a 2-person team to augment the in-house developers as a second phase. The team helped with containerization, re-factoring, and re-architecting the applications to be self-contained and ready for easy deployment and scalability. The team used BitBucket Pipeline to automatically build the code, create an image, and push it to the container registry based on the configuration file in repositories. We used Amazon Elastic Container Registry (ECR) as a Docker container registry to store, share, and deploy container images.

The Continuous Integration (CI) process starts when a programmer commits a change. The pipeline pushes the service (as a Docker image) and the Helm chart (used for package management) to Amazon ECR. An updated container image is deployed to the Kubernetes cluster through the Continuous Deployment (CD) process, where pods containing an image scale automatically based on demand. Avahi team used AWS Cloud Development Kit (CDK) to create the environment, including the Kubernetes clusters. The CDK code created VPC, security groups, and all services on AWS required to set up the Kubernetes environment, including Container registry (ECR), S3 buckets, EKS cluster, AutoScaling groups, and CloudWatch for the logging and monitoring of the services.

Architecture

Below is the architecture diagram of solution designed by Avahi’s cloud solution architects.

Following components were used to build this solution.

Amazon EKS cluster in us-east-2 region. Amazon EKS control plane is fully managed and is deployed in its own VPC.
One VPC with one public subnet and two private subnets in each Availability Zone (AZ) for nodes.
On-Demand instances in an Amazon EC2 Auto Scaling groups (deployed over multiple AZs) that are used as underlying computing infrastructure for the Amazon EKS cluster. EKS offers the possibility of employing multiple ASGs with different instance flavors, where each ASG has specific tags that can be used to schedule pods via label selectors. For our design, we chose C5.9xlarge instances that provide enough compute power for our CPU-hungry applications while also optimizing costs.
Amazon Elastic Container Registry (Amazon ECR) stores the Docker images for application microservices and Helm add-ons for application deployments.
An Amazon Route 53 Domain Name System (DNS) zone for service discovery and a Network Load Balancer configured for HTTPS encrypted traffic.
An Application Load Balancer ingress controller to load balance the application traffic.
Kubernetes Metrics Server to collect metrics from running pods, such as CPU and memory utilization.
Kubernetes Cluster Autoscaler to scale in and out of nodes.
Amazon ElastiCache Redis database that stores the user sessions and state.
Amazon RDS service for SQL databases.
Amazon CloudWatch with Fluent Bit for logging application logs and cluster logs.
Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) and Amazon Simple Storage Service (Amazon S3) for centralized logging.

There are several options available for selecting an ingress controller for EKS, and most of them are open source solutions. Below are the use cases that can influence the decision when selecting ingress controller:

Scaling requirements, available resources, and resource use patterns.
Traffic types served (HTTP, gRPC, WebSocket, etc.)
The number of requests, networking policies, monitoring, and logging requriements.

We decided to use AWS Elastic Load Balancing (ELB) for ingress with a plan to gradually migrate to a more complex solution in the near future. While ELBs may have a limited feature set, it minimizes operational complexity and provides stability and easy scaling.

Timeline of Project

Avahi completed the project in 2 months, including restructuring the code repositories, refactoring the code, containerizing the application, and building CI/CD pipelines.

Business Outcome

With a solid and dependable AWS cloud platform in place, Serviceo focuses on speeding development and implementing a continuous delivery approach. Managing a stable runtime environment with EKS helps the development team realize shorter release cycles and achieve its desired continuous delivery approach. And when rolling out new software, the company appreciates that it doesn’t have to worry about breaking its production environment due to EKS upgrades and failure domains — new releases get deployed smoothly to customers with zero downtime.

The company’s use of AWS services directly impacts the development life cycle, but those improvements benefit the business side as well. Serviceo is reducing the total cost of ownership of its production workloads, creating more flexibility in the business models and pricing structure it can offer its customers. The customer now has an integrated Kustomer / Connect omnichannel contact center solution that can scale as needed while providing the required reporting data.