Adding Envoy NETWORK_FILTER with Client TLS authentication

I am following the Egress gateway for HTTPS traffic (https://istio.io/docs/tasks/traffic-management/egress/egress-gateway/) example and additionally wanted to add a EnvoyFilter at the Egress Gateway to Validate the cert thumbprint.
I have a successfully added a Envoy Lua filter at Ingress gateway to Validate the cert thumbprints, However that EnvoyFilter is of type HTTP_FILTER and Its not effective at Egress Gateway as the traffic goes through Egress gateway is HTTPS. so from my observations so far only NETWORK_FILTER is effective on HTTPS/TLS traffic and Envoy Lua filter does not work for NETWORK_FILTER.

So I have decided to try this Client TLS authentication( https://www.envoyproxy.io/docs/envoy/v1.13.0/api-v2/config/filter/network/client_ssl_auth/v2/client_ssl_auth.proto ) which is NETWORK_FILTER.
Anyone tried this with Istio and got it worked? Or is there any example of how to configure this?

By looking at https://www.envoyproxy.io/docs/envoy/v1.13.0/api-v2/config/filter/network/client_ssl_auth/v2/client_ssl_auth.proto, which has “auth_api_cluster”, “stat_prefix”, “refresh_delay”,“ip_white_list”. It seems that this does not support cert thumbprint validation.

Destination rules could help here https://istio.io/docs/reference/config/networking/destination-rule/#TLSSettings

my attempts so far,

  1. I’ve set up a rest api which gives required json as a responce (Client TLS authentication — envoy 1.29.0-dev-6d9a6e documentation).

  2. Added a service entry as below for adding cluster for “auth_api_cluster”. this will add a cluster name “outbound|80||auth.local”

    apiVersion: networking.istio.io/v1alpha3

    kind: ServiceEntry

    metadata:

    name: auth-local

    spec:

    hosts:

    • auth.local

    location: MESH_INTERNAL

    ports:

    • number: 80

      name: http

      protocol: HTTP

    resolution: STATIC

    endpoints:

    • address: 20.72.25.222
  3. Add a envoy filter as below ,

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: egressgateway-validate-cert
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      app: istio-egressgateway
  configPatches:
    # The first patch adds the 
  - applyTo: NETWORK_FILTER
    match:
      context: ANY
      listener:
        portNumber: 443
        filterChain:
          filter:
            name: "envoy.tcp_proxy"
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.client_ssl_auth
        config:
           auth_api_cluster: outbound|80||auth.local
           stat_prefix: client_ssl_auth

With these settings , I can see it calls the auth endpoint , but it fails with 400 error. debug log from auth pod below,

2020-05-18T21:52:43.166322836Z ':authority', 'outbound|80||auth.local'
2020-05-18T21:52:43.166325836Z ':path', '/v1/certs/list/approved'
2020-05-18T21:52:43.166328636Z ':method', 'GET'
2020-05-18T21:52:43.166331436Z 'x-envoy-internal', 'true'
2020-05-18T21:52:43.166334236Z 'x-forwarded-for', '10.71.65.158'
2020-05-18T21:52:43.166336736Z 'x-envoy-expected-rq-timeout-ms', '1000'
2020-05-18T21:52:43.166339536Z 'content-length', '0'
2020-05-18T21:52:43.166342136Z 
2020-05-18T21:52:43.166731733Z [Envoy (Epoch 0)] [2020-05-18 21:52:43.166][21][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1257] [C1746][S2559221291948996260] request end stream
2020-05-18T21:52:43.16705673Z [Envoy (Epoch 0)] [2020-05-18 21:52:43.166][21][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1354] [C1746][S2559221291948996260] Sending local reply with details invalid_authority
2020-05-18T21:52:43.167355828Z [Envoy (Epoch 0)] [2020-05-18 21:52:43.167][21][debug][filter] [src/envoy/http/mixer/filter.cc:47] Called Mixer::Filter : Filter
2020-05-18T21:52:43.167609426Z [Envoy (Epoch 0)] [2020-05-18 21:52:43.167][21][debug][filter] [src/envoy/http/mixer/filter.cc:148] Called Mixer::Filter : setDecoderFilterCallbacks
2020-05-18T21:52:43.167961423Z [Envoy (Epoch 0)] [2020-05-18 21:52:43.167][21][debug][filter] [src/envoy/http/mixer/filter.cc:135] Called Mixer::Filter : encodeHeaders 0
2020-05-18T21:52:43.168238821Z [Envoy (Epoch 0)] [2020-05-18 21:52:43.168][21][debug][http] [external/envoy/source/common/http/conn_manager_impl.cc:1552] [C1746][S2559221291948996260] encoding headers via codec (end_stream=true):
2020-05-18T21:52:43.168248121Z ':status', '400'
2020-05-18T21:52:43.168257621Z 'date', 'Mon, 18 May 2020 21:52:42 GMT'
2020-05-18T21:52:43.168260721Z 'server', 'istio-envoy'

This “Sending local reply with details invalid_authority” error seems to me like it does not like the
‘:authority’, ‘outbound|80||auth.local’ .

Anyone know how to add a envoy cluster differently so i can get the authority name correctly?