Support Envoy Native Subset Load Balancing

Objective

Support routing HTTP request to a specific backend endpoint through ingress gateway. In our use case, there are some existing tools that monitor k8s Pods. After moving k8s Pods into mesh and Istio mutual TLS being enabled, these tools can’t access Pod anymore and they can’t be run in mesh. The problem can be resolved by making these tools send requests to ingress gateway and routing the request to the specified endpoint.

Replacing current DestinationRule Subset implementation with Envoy subset load balancing is not the goal of this feature request.

Background

Currently Istio generates one Envoy cluster for each subset defined in DestinationRule, backend endpoints are grouped into subset by labels. For using subset, labels are defined in Deployment/Pod spec, subsets are defined in DestinationRule, and route rules are defined in VirtualService, basically everything is defined statically.

It’s not practical to use the DestinationRule subset to route requests to any single specified endpoint without updating DestinationRule/VirtualService spec. If a service has a lot of endpoints(e.g. >1000), the size of RDS/CDS/EDS config will increase dramatically.

Requirements

In our use case, client needs to send HTTP requests to a specific endpoint of a service through ingress gateway, the destination endpoint can be specified by a HTTP header. This feature can be enabled for specific virtual hosts or route, e.g. only allowed for host foo.example.com for URI path `/foo’. When a new Pod is created or deleted, there is no need to update DestinationRule/VirtualService for adding new subset/route rule or clean up old subset/route rule.

Design Ideas

Envoy’s Subset Load Balancing comes in handy for this use case, endpoint can be generated with metadata in namespace envoy.lb, then Envoy Header-To-metadata filter can be used to transform header into metadata in namespace envoy.lb. In this way, the destination endpoint can be selected through a header, no need to update DestinationRule/VirtualService.

In our use case, an endpoint’s IP address can be used as metadata to select the endpoint. For adding its IP address as a metadata, a replacement pattern %ENDPOINT_IP% can be used, the DestinationRule spec could be:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
spec:
  subsetLB: # Envoy subset LB config
    metadata:
        endpoint-ip: %ENDPOINT_IP%  # in metadata namespace `envoy.lb` 
    fallbackPolicy: NO_FALLBACK
  subsets:
  - name: v1
    subsetLB: # this can also be applied to DestinationRule Subset
      metadata:
          endpoint-ip: %ENDPOINT_IP% 
      fallbackPolicy: NO_FALLBACK

The generated endpoint config will be:

{
    "endpoint": {
        "address": {
            "socket_address": {
                "address": "192.168.0.100",
                "port_value": 8080
            }
        }
    },
    "metadata": {
        "filter_metadata": {
...
            "envoy.lb": {
                "endpoint-ip": "192.168.0.100"
            }
        }
    },
...
}

The corresponding subset LB config in Envoy Cluster should be:

{
    "cluster": {
 ...
        "lb_subset_config": {
            "subset_selectors": [
                {
                    "keys": [
                        "endpoint-ip"
                    ],
                    "single_host_per_subset": true
                }
            ]
        }
    }
}

A sample VirtualService for adding metadata to HTTP requests match /foo could be:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
...
spec:
  hosts:
  - foo.example.com
  http:
  - match:
    - uri:
        prefix: "/foo"
      headers:
        request:
...
          subsetLb:  # transform header into metadata namespace `envoy.lb` 
          - header: destination-endpoint 
            on_header_present:# same as header_to_metadata KeyValuePair,
                              # but no metadata_namespac field, so user
                              # can’t add metadata to other namespace
              key: endpoint-ip
              type: STRING
            remove: false
...

The generated Header-To-Metadata filter config will be :

route_config:
  virtual_hosts:
  - domains:
    - "foo.example.com"
    routes:
    - match:
        prefix: "/foo"
      typed_per_filter_config:
        envoy.filters.http.header_to_metadata:
          "@type": "type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config"
          request_rules:
          - header: instance
            on_header_present:
              metadata_namespace: envoy.lb
              key: instance
              type: STRING
            remove: false