Exploring Istio

As modern software becomes more complex, large monoliths are broken down to smaller parts; they are commonly known as microservices.

Microservices offer many benefits, they allow engineers more flexibility in terms of technology stack and each service can scale horizontally or vertically depending on their needs. However, there are also some disadvantages to namely

• Error Handling
• Routing
• Service Discovery
• Observability

Error Handling

In a microservice architecture, services have to communicate with one another. For example, a Video service has to get subititles from the Subtitle Service. However, the subtitle service might fail due to not having enough memory or face delays in generating subtitles. As the Video service, how should it handle the failure ?

• How long does it have to wait for a response ?
• How many retries should it attempt without overwhelming the Video service ?

Routing

Some services might require more fine-grained requirements, such as traffic shifting (i.e upgrading from an old version to a new version - directing 20% of the traffic)

Service Discovery

Microservices scale up, down or fail at any time. As a result, IP addresses changes frequently, it becomes hard to maintain these changes

Observability

Lastly, the more services there are, the greater the need to have a centralise solution to collect metrics such as Latency, Traffic, Error and Saturation (4 Golden Signals)

Introducing Istio

Istio is an open-sourced service mesh designed by Google & Lyft. The main idea of Istio is to provide a proxy that sits in front of every service; also known as side car. The side car allows us to define networking rules and help us gather metrics. Istio can be used for blue green deployment, retries, timeout, JWT, mTLS, modifying response headers, load balancing etc.

tldr; Istio enables east-west communication (service to service)

Below is the architecture diagram for Istio.

So how does a request flow in Istio ?

1. External ingress traffic (HTTP, TCP) reaches the cluster.
2. Traffic is routed to istiod (istio-daemon). Istiod is responsible for

• Service Discovery (Knows where all the services are)
• Applying traffic management rules on the Envoy (Timeouts, Retries, Circuit breakeres, Fault Injection)
• Certificate management for secure communications

3. The request is routed back to the Data plane, where one of the Proxy (Envoy) will receive the request.
4. Request will be forwarded from Envoy -> Service A.
5. Some business logic will take place in Service A.
6. If Service A has to communicate with Service B, A will forward traffic back to its own proxy. Using service discovery and traffic management rules, Proxy A can determine the location of Service B.
7. Throughout this flow, the Proxy is reporting telemetry data to istiod.

Istio has many use cases such as traffic shifting, traffic mirroring, modify response headers. Futhermore, you can install an addon called Kiali, which provides the UI to help us understand how traffic in our service mesh.

Below is a snippet of how we can shift traffic

// traffic-shifting.yaml
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: gateway
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "*"
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: app
spec:
  host: app
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: app
spec:
  gateways:
  - gateway
  hosts:
  - "*"
  http:
  - match:
    - uri:
        prefix: /myapp
    rewrite:
      uri: /
    route:
    - destination:
        host: app.test.svc.cluster.local
        subset: v1
        port:
          number: 8080
      weight: 90
    - destination:
        host: app.test.svc.cluster.local
        subset: v2
        port:
          number: 8080
      weight: 10

More examples can be found in my GitHub repository.

That's all for now. See you in the next one !

Here are some useful links that helped me.

GitHub Repository: https://github.com/kaikiat/istio-tutorial

Istio By Example: https://istiobyexample.dev/