Minimizing complexity with service mesh lite architecture
Submitted January 14, 2021
Author: Priyanka Sharma
As more organizations adopt microservice architectures, it has become critical to manage, monitor, and secure service-to-service (east-west) communication. A service mesh is a dedicated infrastructure layer that tracks, secures, and relays all data flow between services and is usually implemented by deploying network proxies as sidecars in the data plane.
Popular service mesh solutions like Istio or LinkerD solve many of the challenges of service-to-service communication, but things can get complex quickly because of the presence of many data plane proxies that are used to connect disparate containers in the cluster. Also, service mesh solutions can create more resource overhead by having a control plane and one proxy for each microservice.
A service mesh lite architecture gives you the benefits of service mesh, all while minimizing complexity by using fewer proxies in the deployment. Learn more about the philosophy behind service mesh lite architecture and its benefits in this blog post. Here, we’ll take a look at how you can use NetScaler ADCs to run your application in a service mesh lite architecture.
Benefits of Service Mesh Lite
An ingress solution (hardware, virtualized, or containerized) typically performs L7 proxy functions for north-south traffic. The service mesh lite architecture uses the same ingress solution to manage east-west traffic, as well.
In Kubernetes, east-west traffic traverses the built-in KubeProxy deployed in each node. As an L4 proxy, KubeProxy only does TCP/UDP-based load balancing without the benefits of L7 proxy. However, NetScaler ADCs (MPX, VPX, or CPX) can provide other benefits for east-west traffic, including:
SSL offload
Content-based routing, allow/block traffic based on HTTP, and HTTPS header parameters
Advanced load balancing algorithms (least connections, least response time)
Observability of east-west traffic through measuring golden signals like errors, latencies, saturation, and traffic volume (NetScaler ADM’s service graph is an observability solution to monitor and debug microservices.)
Understanding Service Mesh Lite
Citrix Ingress Controller is built around Kubernetes ingress and automatically configures one or more NetScaler ADCs based on the ingress resource configuration present in a Kubernetes cluster. (We have resources available if you need a refresher on Citrix Ingress Controller and NetScaler ADC form factors.)
Let’s look at service mesh lite and how you can run your application with this architecture using NetScaler ADC CPX/VPX and Citrix Ingress Controller in your Kubernetes cluster.
East-West Communication in Kubernetes
Let’s consider an e-commerce application — Hipster — that has multiple microservices. Here is the manifest that can be used to deploy the Hipster application in Kubernetes. We’ll look at two microservices in the application — cartservice and checkoutservice — to understand how kube-proxy handles east-west traffic. When the cartservice microservice wants to access the checkoutservice microservice, the DNS returns the cluster IP of the checkoutservice microservice to the cartservice microservice. The cartservice microservice initiates a connection to the cluster IP, which the KubeProxy intercepts. The KubeProxy load balances the request between the set of checkoutservice pods.
E-W traffic to checkoutservice app via KubeProxy
With service mesh lite, we introduce a proxy in the east-west traffic path to leverage its features with the east-west application traffic in a Kubernetes cluster. Service mesh lite leverages the headless service concept instead of the cluster IP in Kubernetes. Kube-proxy does not handle headless services, and there’s no load balancing or proxying done for them by the platform.
In the next sections, we’ll look at two cases to help you understand how to use the headless service concept of Kubernetes to introduce NetScaler ADC into the east-west traffic path so that it can control and load balance the traffic between various microservices running in a Kubernetes cluster.
Case I: East-West Communication with NetScaler ADC CPX in Service Mesh Lite Architecture
Our goal here is to insert the NetScaler ADC CPX in the east-west traffic path and use the Kubernetes Ingress rules to control the traffic between various microservices in the application.
Step 1: Modify the checkoutservice service definition to point to NetScaler ADC CPX
We want all traffic destined for the checkoutservice microservice to land in the CPX. To achieve this, we want the checkoutservice DNS to be resolved to the IP of the CPX. You can do this by making service of checkoutservice microservice point to the CPX deployment. In the service manifest of checkoutservice, we specify the selector as a label given in the NetScaler ADC CPX deployment manifest. In this case it’s ‘app:cpx-ingress1’.
E-W traffic to checkoutservice app via NetScaler ADC CPX
After this modification, when a pod in the Kubernetes cluster resolves the FQDN for service of the checkoutservice microservice, the IP address of the NetScaler ADC CPX is returned.
Step 2: Create the headless service checkoutservice-headless for checkoutservice microservice pods
Because we have modified the original service of checkoutservice microservice so that it points to the NetScaler ADC CPX IP, we need to create one more service that represents the checkoutservice microservice deployment. The headless service will look like:
apiVersion: v1kind: Servicemetadata: name: checkoutservice-headlessspec: selector: name: checkoutservice ports: - name: grpc port: 5050 targetPort: 5050 #headless Service clusterIP: None
Step 3: Create an ingress resource with rules for checkoutservice-headless service
We need to have an ingress resource for the checkoutservice microservice that can be used by the Citrix Ingress Controller to configure the NetScaler ADC CPX so it can handle all the east-west traffic destined to the checkoutservice microservice.
A sample ingress resource (for checkoutservice) is shown here:
apiVersion: extensions/v1beta1kind: Ingressmetadata: name: checkoutservice-ingress annotations: ingress.citrix.com/insecure-service-type: "tcp" ingress.citrix.com/insecure-port: "5050" kubernetes.io/ingress.class: "tier-2-cpx1"spec: backend: serviceName: checkoutservice-headless servicePort: 5050
With the above changes for all the application’s microservices, all traffic coming to a microservice will land on the NetScaler ADC CPX first. It can then load balance the east-west traffic in the cluster.
The following diagrams show the end-to-end flow of how the NetScaler ADC CPX can be introduced in the east-west traffic path in the Kubernetes cluster so that it can provide L7 proxying communication between the cartservice and checkoutservice microservices using ingress rules.
The following diagram shows the NetScaler ADC CPX configured as an east-west proxy using ingress rules.
This is a dual-tier topology. You can use this manifest to deploy the Hipster application in a service mesh lite architecture in the dual-tier topology. You will have make sure that you provide the following information in the manifest before you deploy it in the Kubernetes cluster:
<citrix-adc-vpx-nsip>: The management IP address of a standalone NetScaler ADC appliance. For more information, see IP Addressing in NetScaler ADC.
<citrix-adc-vpx-credential-secret>: Name of the Kubernetes Secret having ADC credential details.
<citrix-adc-vpx-vip>: Citrix Ingress Controller configures this virtual IP address in the NetScaler ADC to receive ingress traffic.
Learn more about dual-tier topology and the information required to bring up the topology.
Once you can successfully deploy this manifest and everything is up and running in the tier-1 NetScaler ADC VPX and the tier-2 NetScaler ADC CPX, you will be able to access the application by following these steps:
Enter the application hostname in your host file against the NetScaler ADC VPX VIP:x.x.x citrix.hipster.smlite.
You should be able to access the application via http://citrix.hipster.smlite in your browser.
In this deployment all the east-west traffic is handled and load-balanced by tier-2 NetScaler ADC CPX.
Case II: East-West Communication with NetScaler ADC MPX or VPX in Service Mesh Lite Architecture
Similarly (but with slight modifications), you can use NetScaler ADC MPX or VPX, acting as an ingress, to load balance east-west microservice communication. Here’s how:
Step 1: Create an external service resolving the checkoutservice host name to the NetScaler ADC MPX/VPX IP address
There are two ways to achieve this. You can add a service of type ExternalName by mapping a host name or using an IP address.
Mapping a host name (CNAME)
Create a domain name for the ingress endpoint IP (content switching virtual server IP address) in NetScaler ADC MPX/VPX — myadc.mydomain.com, for example — and update it in your DNS server.
Create a Kubernetes service for checkoutservice with externalName as myadc.mydomain.com.
Now, when any pod looks up the checkoutservice microservice, a CNAME(myadc.mydomain.com) is returned.
apiVesion: v1kind: Servicemetadata: name: checkoutservicespec: type: ExternalName externalName: myadc.mydomain.com
Mapping a host name to an IP address
When you want your application to use the host name checkoutservice, which will redirect to the virtual IP address hosted in NetScaler ADC MPX or VPX, you can use the following.
apiVersion: v1kind: Servicemetadata: name: checkoutservicespec: ports: - name: grpc port: 5050 targetPort: 5050---apiVersion: v1kind: Endpointsmetadata: name: checkoutservicesubsets: - addresses: - ip: "1.1.1.1" # Ingress IP in VPX/MPX ports: - port: 5050 name: grpc
Step 2: Create a headless service checkoutservice-headless for the checkoutservice microservice pods
Because we have modified service of checkoutservice microservice to point to NetScaler ADC MPX, we need to create one more service that represents the checkoutservice microservice deployment.
The headless service for checkoutservice microservice will look like:
apiVersion: v1kind: Servicemetadata: name: checkoutservice-headlessspec: selector: app: checkoutservice ports: - name: grpc port: 5050 targetPort: 5050 #headless Service clusterIP: None
Step 3: Create an ingress resource with rules for checkoutservice-headless service having the ingress.citrix.com/frontend-ip annotation
Create an ingress resource with the ingress.citrix.com/frontend-ip annotation, where the value matches the ingress endpoint IP address in NetScaler ADC MPX or VPX. The Citrix Ingress Controller will use this ingress resource to configure the NetScaler ADC VPX for handling traffic coming to the microservice checkoutservice.
Here’s a sample ingress resource:
apiVersion: extension/v1beta1kind: Ingressmetadata: name: checkoutservice-ingress annotations: ingress.citrix.com/insecure-service-type: "tcp" ingress.citrix.com/insecure-port: "5050" kubernetes.io/ingress.class: "tier-2-cpx1" ingress.citrix.com/frontend-ip: "1.1.1.1"spec: backend: serviceName: checkoutservice-headless servicePort: 5050
With the above changes in the microservices of an application, all traffic coming to a microservice will land on the NetScaler ADC MPX/VPX first. Using the usual ingress load balancing methodology with the changes above, the NetScaler ADC VPX/MPX can now load balance east-west traffic.
The following diagram shows NetScaler ADC VPX or MPX configured as north-south and east-west proxies using ingress rules.
The above topology is a unified-ingress or single-tier topology. This manifest can be used to deploy the Hipster application in a service mesh lite architecture in the unified-ingress topology. You must make sure that you provide the following information in the manifest before you deploy it in the Kubernetes cluster:
<citrix-adc-vpx-nsip>: The management IP address of a standalone NetScaler ADC appliance. For more information, see IP Addressing in NetScaler ADC.
<citrix-adc-vpx-credential-secret>: Name of the Kubernetes secret having ADC credential details.
<citrix-adc-vpx-vip>: Citrix Ingress Controller configures this virtual IP address in the NetScaler ADC to receive ingress traffic.
Learn more about unified ingress topology and the information required to bring up the topology.
Once you successfully deploy the above manifest and everything is up and running in tier-1 NetScaler ADC VPX, you will be able to access the application by following these steps:
Enter the application hostname in your host file against NetScaler ADC VPX VIP:x.x.x citrix.hipster.smlite
You should be able to access the application by accessing http:/citrix.hipster.smlite in your browser
In this deployment all the east-west traffic is handled and load-balanced by tier-1 NetScaler ADC VPX.
Automated Deployment of Apps in Service Mesh Lite
To deploy an application in a service mesh lite architecture, you’ll need to perform multiple tasks, including:
Modifying the existing services to make them headless services
Creating a service to point to NetScaler ADC
Creating ingress rules
NetScaler provides an automated way to generate deployment-ready YAMLs to make deploying a service mesh lite solution easier. Learn more.
Conclusion
Service mesh lite architecture can give you the benefits of service mesh while minimizing complexity. You can run your app in a service mesh lite architecture using NetScaler ADC CPX or VPX/MPX and Citrix Ingress Controller in your Kubernetes cluster. NetScaler ADCs handle the east-west traffic of your application in the service mesh lite architecture and, as an L7 proxy, can provide features other proxies can’t.
To learn more about using Citrix Ingress Controller to enable different features on NetScaler ADC CPX, VPX, and MPX, check out our GitHub repository.
