EnvoyFilter not working with dex+oauth2-proxy to authenticate cluster UIs (k8s-dashboard/grafana-dashboard)

Networking
1

Hello,

We are implementing ISTIO on top of AWS EKS cluster. Gateway and respective virtual services are working. In this environment we are using helm to deploy all the cluster component including Istio(using templates).
Our next goal is to authenticate the cluster UIs using dex & oauth2-proxy(this solution works and running in our legacy cluster with nginx). Unfortunately with the following config it’s not working with Istio.

EKS Version : 1.20
Helm Version: 3.5.3
Kubectl Version: 1.21.0(client)/1.20.7(server)
Istio Version: 1.11.1

Dex configuration template#

grpc: false
certs:
  grpc:
    create: false
  web:
    create: false
ports:
  web:
    servicePort: 5556
  telemetry:
    servicePort: 5558
config:
  issuer: https://dex.example.com
  connectors:
  - type: mockCallback
    id: mock
    name: <cluster_name>
  - type: <connector_type>
    id: <connector_id>
    name: <connector_name>
    config:
      issuer: <issuer>
      clientID: <client_id>
      clientSecret: <client_secret>
      redirectURI: https://dex.example.com/callback
      scopes: ["openid","profile","email","groups","address","phone","full"]
  enablePasswordDB: true
  staticClients:
  - id: 'oidc-auth-client'
    redirectURIs:
    - "https://auth.example.com/oauth2/callback"
    - "https://dashboard.example.com/oauth2/callback"
    - "https://grafana.example.com/oauth2/callback"
    name: 'oidc-auth-client'
    secret: KYCI4XWYZGhu8cAC0hq0xtf1XWcQxPBU1HyzOZdGxi8=
  staticPasswords:
  - email: "admin@example.com"
    hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"
    username: "admin"
    userID: "08a8684b-db88-4b73-90a9-3cd1661f5466"
  telemetry:
    http: 0.0.0.0:5558
  frontend:
    theme: coreos
ingress:
  enabled: false
service:
  type: ClusterIP
livenessProbe:
  enabled: true
  initialDelaySeconds: 1
  failureThreshold: 1
  httpPath: "/healthz"
  periodSeconds: 10
  timeoutSeconds: 5

Oauth2-proxy configuration template#

service:
  type: ClusterIP
  port: 4180
config:
  clientID: 'oidc-auth-client'
  clientSecret: KYCI4XWYZGhu8cAC0hq0xtf1XWcQxPBU1HyzOZdGxi8=
  cookieSecret: KYCI4XWYZGhu8cAC0hq0xtf1XWcQxPBU1HyzOZdGxi8=
extraArgs:
  provider: oidc
  provider-display-name: dex
  proxy-websockets: true
  oidc-issuer-url: https://dex.example.com
  cookie-secure: true
  cookie-name: auth
  cookie-refresh: 1h
  cookie-expire: 4h
  cookie-httponly: true
  email-domain: "*"
  pass-host-header: true
  ping-path: /ping
  set-authorization-header: true
  skip-provider-button: true
  http-address: 0.0.0.0:4180
  upstream: static://200
  scope: openid profile email offline_access groups
  client-secret: KYCI4XWYZGhu8cAC0hq0xtf1XWcQxPBU1HyzOZdGxi8=
  client-id: 'oidc-auth-client'
  whitelist-domain: .example.com
  cookie-domain: .example.com
ingress:
  enabled: false

EnvoyFilter configuration#

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: kubernetes-dashboard
  namespace: istio-system
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: envoy.http_connection_manager
            subFilter:
              name: istio.metadata_exchange
          sni: dashboard.example.com
    patch:
      operation: INSERT_AFTER
      value:
        name: envoy.filters.http.ext_authz
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
          http_service:
            authorizationRequest:
              allowedHeaders:
                patterns:
                - exact: accept
                - exact: authorization
                - exact: cookie
                - exact: from
                - exact: proxy-authorization
                - exact: user-agent
                - exact: x-forwarded-access-token
                - exact: x-forwarded-email
                - exact: x-forwarded-for
                - exact: x-forwarded-host
                - exact: x-forwarded-proto
                - exact: x-forwarded-user
                - prefix: x-auth-request
                - prefix: x-forwarded
            authorizationResponse:
              allowedClientHeaders:
                patterns:
                - exact: authorization
                - exact: location
                - exact: proxy-authenticate
                - exact: set-cookie
                - exact: www-authenticate
                - prefix: x-auth-request
                - prefix: x-forwarded
              allowedUpstreamHeaders:
                patterns:
                - exact: authorization
                - exact: location
                - exact: proxy-authenticate
                - exact: set-cookie
                - exact: www-authenticate
                - prefix: x-auth-request
                - prefix: x-forwarded
            server_uri:
              cluster: outbound|4180||oauth2-proxy.auth.svc.cluster.local
              timeout: 1.5s
              uri: http://oauth2-proxy.auth.svc.cluster.local

When we are trying to access the k8s-dashboard (https://dashboard.example.com), it’s redirecting to https://dashboard.example.com/#/overview?namespace=default.
There is no log generating in oauth2-proxy for this action.
From istio-ingressgateway pod we found the following log…

{"level":"debug","time":"2021-10-26T07:58:22.577962Z","scope":"envoy router","msg":"[C110][S10772054866034720886] cluster 'outbound|80||kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local' match for URL '/api/v1/serviceaccount/default?itemsPerPage=10&page=1&sortBy=d,creationTimestamp'"}

Looking for guidance from the community.
Helps/Suggestions are welcome to fix this issue.