Proxy ingress traffic to a specific pod

Hi All,

I have a unique problem. I have a downstream system who would like to connect to my k8s cluster of course using Istio as the mesh. But this connection (TCP) has to be proxied untouched to a specific pod (which is dynamic per request) in the cluster. The reason being, this specific pod is connected via websocket to an upstream system. In essence, the downstream system should just proxy the connection via mesh, this pod to the upstream system.

From what I have researched, it seems under Network filters, TcpProxy along with the original destination cluster maybe the one to use.

Below are the configs I have tried with no luck.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: orig-dest-filter
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: NETWORK_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: envoy.filters.listener.original_dst
      patch:
        operation: INSERT_BEFORE
        value:
          name: "envoy.filters.network.tcp_proxy"
          typed_config:
            '@type': type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
            cluster: original_destination_cluster
    - applyTo: HTTP_FILTER
      match:
        context: ANY
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
              subFilter:
                name: envoy.filters.http.router
      patch:
        operation: MERGE
        value:
          name: "envoy.filters.network.http_connection_manager"
          typed_config:
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
            stat_prefix: http
            route_config:
              name: route_a
              virtual_hosts:
              - name: route_a
                domains:
                - "*"
                routes:
                - match:
                    prefix: /
                  route:
                    cluster: original_destination_cluster
    - applyTo: CLUSTER
      patch:
        operation: ADD
        value: # cluster specification
          name: original_destination_cluster
          type: ORIGINAL_DST
          original_dst_lb_config:
            use_http_header: true
          connect_timeout: 6s
          lb_policy: CLUSTER_PROVIDED
          dns_lookup_family: V4_ONLY

Proxy config details, after creating Gateway object

hydraulics@hydraulics-virtual-machine:~$ istioctl proxy-config all istio-ingressgateway-77968dbd74-l2bft.istio-system
SERVICE FQDN                                            PORT      SUBSET     DIRECTION     TYPE             DESTINATION RULE
BlackHoleCluster                                        -         -          -             STATIC
agent                                                   -         -          -             STATIC
hello-world.test.svc.cluster.local                      8080      -          outbound      EDS
istio-egressgateway.istio-system.svc.cluster.local      80        -          outbound      EDS
istio-egressgateway.istio-system.svc.cluster.local      443       -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     80        -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     443       -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     15021     -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     15443     -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     31400     -          outbound      EDS
istiod.istio-system.svc.cluster.local                   443       -          outbound      EDS
istiod.istio-system.svc.cluster.local                   15010     -          outbound      EDS
istiod.istio-system.svc.cluster.local                   15012     -          outbound      EDS
istiod.istio-system.svc.cluster.local                   15014     -          outbound      EDS
kube-dns.kube-system.svc.cluster.local                  53        -          outbound      EDS
kube-dns.kube-system.svc.cluster.local                  9153      -          outbound      EDS
kubernetes.default.svc.cluster.local                    443       -          outbound      EDS
metrics-server.kube-system.svc.cluster.local            443       -          outbound      EDS
original_destination_cluster                            -         -          -             ORIGINAL_DST
prometheus_stats                                        -         -          -             STATIC
sds-grpc                                                -         -          -             STATIC
se-host                                                 80        -          outbound      EDS
xds-grpc                                                -         -          -             STATIC
zipkin                                                  -         -          -             STRICT_DNS

ADDRESS PORT  MATCH DESTINATION
0.0.0.0 8080  ALL   Route: http.8080
0.0.0.0 15021 ALL   Inline Route: /healthz/ready*
0.0.0.0 15090 ALL   Inline Route: /stats/prometheus*

NAME          DOMAINS     MATCH                  VIRTUAL SERVICE
http.8080     *           /*                     404
              *           /stats/prometheus*
              *           /healthz/ready*

Using a custom listener instead of using Gateway context. The Cluster and route seems to be created but the request just waits forever.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: orig-dest-filter
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: LISTENER
      match:
        context: ANY
      patch:
        operation: ADD
        value:
          name: proxy
          # traffic_direction: INBOUND
          address:
            socket_address:
              protocol: TCP
              address: 0.0.0.0
              port_value: 8080
          filter_chains:
          - filters:
            - name: "envoy.filters.network.http_connection_manager"
              typed_config:
                "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
                stat_prefix: ingress_proxy
                route_config:
                  name: route_a
                  virtual_hosts:
                  - name: route_a
                    domains:
                    - "*"
                    routes:
                    - match:
                        prefix: "/abc"
                      route:
                        cluster: original_destination_cluster
    - applyTo: CLUSTER
      patch:
        operation: ADD
        value: # cluster specification
          name: original_destination_cluster
          type: ORIGINAL_DST
          original_dst_lb_config:
            use_http_header: true
          connect_timeout: 6s
          lb_policy: CLUSTER_PROVIDED
          dns_lookup_family: V4_ONLY

proxy config details

$ istioctl proxy-config all istio-ingressgateway-77968dbd74-l2bft.istio-system
SERVICE FQDN                                            PORT      SUBSET     DIRECTION     TYPE           DESTINATION RULE
BlackHoleCluster                                        -         -          -             STATIC
agent                                                   -         -          -             STATIC
hello-world.test.svc.cluster.local                      8080      -          outbound      EDS
istio-egressgateway.istio-system.svc.cluster.local      80        -          outbound      EDS
istio-egressgateway.istio-system.svc.cluster.local      443       -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     80        -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     443       -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     15021     -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     15443     -          outbound      EDS
istio-ingressgateway.istio-system.svc.cluster.local     31400     -          outbound      EDS
istiod.istio-system.svc.cluster.local                   443       -          outbound      EDS
istiod.istio-system.svc.cluster.local                   15010     -          outbound      EDS
istiod.istio-system.svc.cluster.local                   15012     -          outbound      EDS
istiod.istio-system.svc.cluster.local                   15014     -          outbound      EDS
kube-dns.kube-system.svc.cluster.local                  53        -          outbound      EDS
kube-dns.kube-system.svc.cluster.local                  9153      -          outbound      EDS
kubernetes.default.svc.cluster.local                    443       -          outbound      EDS
metrics-server.kube-system.svc.cluster.local            443       -          outbound      EDS
original_destination_cluster                            -         -          -             STATIC
prometheus_stats                                        -         -          -             STATIC
sds-grpc                                                -         -          -             STATIC
se-host                                                 80        -          outbound      EDS
xds-grpc                                                -         -          -             STATIC
zipkin                                                  -         -          -             STRICT_DNS

ADDRESS PORT  MATCH DESTINATION
0.0.0.0 8080  ALL   Inline Route: /abc*
0.0.0.0 15021 ALL   Inline Route: /healthz/ready*
0.0.0.0 15090 ALL   Inline Route: /stats/prometheus*

NAME        DOMAINS     MATCH                  VIRTUAL SERVICE
            *           /stats/prometheus*
            *           /healthz/ready*
route_a     *           /*
route_a     *           /abc*

Any direction is greatly appreciated. If my use case is something that can’t be solved using the above approach, please let me know.