JWT Policy does not take affect!

SangNguyen · January 11, 2019, 5:38am

Hi Everyone,

I’ve been applying Authentication Policy to my testing service using JWT. I have followed the guide on this link: https://istio.io/docs/tasks/security/authn-policy/#end-user-authentication. And yes, it did work as expected. But when I tried to using a different pod image, it did not work even though almost everything is the same. Is there anyone facing this issue? And know the reason why it did not work in my case? Thank you very much!

Here is the full yaml content:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hostname
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hostname
      version: v1
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: hostname
        version: v1
    spec:
      containers:
      - image: rstarmer/hostname:v1
        imagePullPolicy: Always
        name: hostname
        resources: {}
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: hostname
  name: hostname
spec:
  ports:
  - name: http
    port: 8001
    targetPort: 80
  selector:
    app: hostname
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: hostname-gateway
  namespace: foo
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
piVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hostname-vs
  namespace: foo
spec:
  hosts:
  - "*"
  gateways:
  - hostname-gateway
  http:
  - route:
    - destination:
        port:
          number: 8001
        host: hostname.foo.svc.cluster.local
---
apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: "jwt-example"
  namespace: foo
spec:
  targets:
  - name: hostname
  origins:
  - jwt:
      issuer: "testing@secure.istio.io"
      jwksUri: "https://raw.githubusercontent.com/istio/istio/release-1.0/security/tools/jwt/samples/jwks.json"
  principalBinding: USE_ORIGIN

YangminZhu · January 11, 2019, 7:06pm

Which namespace are you deploying the hostname service? It should be in the same namespace as the jwt policy.

SangNguyen · January 12, 2019, 2:15am

Hi @YangminZhu,
Yes, I did deploy to namespace ‘foo’ for the hostname. I forgot to put it into the yaml above. Everything will be deployed into namespace ‘foo’. But it did not work!

YangminZhu · January 14, 2019, 8:15pm

Could you try to follow the guide here: https://preliminary.istio.io/help/ops/security/end-user-auth/

Most importantly, could you attach the debug logs from Pilot and the Istio proxy (envoy)?

What do you mean it did not work? Is the request with the JWT token being rejected? or a request without JWT token being accepted?

SangNguyen · January 14, 2019, 9:03pm

Hi @YangminZhu,

I have tried to use another image to test it and it’s working now.

What do you mean it did not work? Is the request with the JWT token being rejected? or a request without JWT token being accepted?
What I meant it did not work means the JWT policy did not take affect, the request still returns status of 200. Like there is no policy applied to the service.

I did not know why it did not affect with the image which I used in the yaml above. Sorry, I did refresh the cluster and use a different image, so I don’t have the log right now but as far as I remembered the Istio-proxy logs is normal, I don’t know how to get the Pilot logs yet (please guide me if you know).

Thanks!

apolegoshko · January 24, 2019, 2:25pm

Hi SangNguyen, did you change the image of your deployment ?

SangNguyen · January 24, 2019, 3:47pm

Hi @apolegoshko,

Yes, I did change to another image and it was working well.

Julien_Senon · January 24, 2019, 7:06pm

You can also check log on POD Istio-telemetry and container mixer

procyclinsur · March 19, 2019, 7:58am

I am having the same issue! I checked out the link that @YangminZhu sent and there was a discrepancy in step 4 in that there was no local_jwks set.

It is useful to know what the issue is but might be better if the documentation described how to fix it. Maybe an unknown issue?

I have listed my environment below after reapplying it.

My Environment:

[procyclinsur@P-428 k8s]$ for i in `ls`; do kubectl apply -n solarmori -f $i ; done
configmap/core-api-config-map created
deployment.extensions/core-api-app created
gateway.networking.istio.io/core-api-gateway created
virtualservice.networking.istio.io/core-api-virtual-service created
policy.authentication.istio.io/core-api-policy created
service/core-api-service created

My Policy:

apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
  name: core-api-policy
  namespace: solarmori
spec:
  targets:
  - name: core-api-service
  origins:
  - jwt:
      issuer: "https://cognito-idp.ap-northeast-1.amazonaws.com/ap-northeast-1_pa9vj7sbL"
      audiences:
      - "72d1h5ovjsgsv595ob9kjntov0"
      jwksUri: "https://cognito-idp.ap-northeast-1.amazonaws.com/ap-northeast-1_pa9vj7sbL/.well-known/jwks.json"
  principalBinding: USE_ORIGIN

My Service:

apiVersion: v1
kind: Service
metadata:
  name: core-api-service
spec:
  type: LoadBalancer
  ports:
  - port: 80
    name: api-svc-port
    targetPort: api-app-port
  selector:
    app: core-api-app

core-api-app istio-proxy log:

[procyclinsur@P-428 istio]$ kubectl logs -n solarmori core-api-app-5dd9666777-qhf5v -c istio-proxy | grep local_jwks
[procyclinsur@P-428 istio]$

If anyone knows the solution I would be grateful!

Guilherme_Durazzo · March 21, 2019, 1:39pm

I am having the same issue TOO",
I can access this service normally, as if there was no policy

my policy

apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: "example-3"
  namespace: "eva"
spec:
  targets:
  - name: broker-eva
  peers:
  - mtls:
  origins:
  - jwt:
      issuer: "https://keycloak.eva.bot/auth/realms/eva.bot"
      jwksUri: "https://keycloak.eva.bot/auth/realms/eva.bot/protocol/openid-connect/certs"
  principalBinding: USE_ORIGIN

my service

apiVersion: v1
kind: Service
metadata:
  name: broker-eva
  namespace: eva
  labels:
    app: broker-eva
spec:
  ports:
  - port: 8080
    name: http
  selector:
    app: broker-eva
---

chaturvedia · March 21, 2019, 6:10pm

For Policy I have mTLS as below
peers:

mtls: {}

Service port name has prefix http-

And it works fine. I have mTLS turned on

Guilherme_Durazzo · March 21, 2019, 7:03pm

I put mtls: {}, but it is still not working, my access still enabled,

and i dont understand this part " Service port name has prefix http-"

SangNguyen · March 21, 2019, 7:47pm

Hi @Guilherme_Durazzo,

It fixed my problem. Try to apply the naming convention as follow: https://istio.io/help/faq/traffic-management/#naming-port-convention.

Guilherme_Durazzo · March 21, 2019, 8:53pm

@SangNguyen yes, i tried and i followed this tutorial too https://istio.io/docs/tasks/security/authn-policy/#end-user-authentication and nothing yet .

chaturvedia · March 22, 2019, 6:23pm

As @SangNguyen mentioned there are some restrictions on port naming https://istio.io/help/faq/traffic-management/#naming-port-convention .

sedgewickmm18 · May 28, 2019, 8:25am

@Guilherme_Durazzo have you found out and fixed the issue in the meantime ?
I have the same problem although policy is in the same namespace, target is the service name ‘letsencryptverifier’ and the service’s port is called http-letsencrypt.
There is a default policy for mutual TLS active and the specific policy for testing JWT auth

spec:
  origins:
  - jwt:
      issuer: testing@secure.istio.io
      jwksUri: https://raw.githubusercontent.com/istio/istio/release-1.1/security/tools/jwt/samples/jwks.json
  peers:
  - {}
  principalBinding: USE_ORIGIN
  targets:
  - name: letsencryptverifier

Thanks for any hint

YangminZhu · May 28, 2019, 8:29pm

There is a default policy for mutual TLS active and the specific policy for testing JWT auth

Are you having 2 authentication policies for the same service? (i.e. 2 different Policy with the same target letsencryptverifier). If so, could you remove any extra Policy and make sure there is only 1 Policy selecting the target letsencryptverifier?

YangminZhu · May 28, 2019, 8:36pm

This seems indicating Pilot somehow failed to get the JWT public key at jwksUri. Would you mind to check the pilot logs and see if there is anything related to pubkey or jwt?

sedgewickmm18 · May 29, 2019, 4:18am

I’ve deleted the default policy and edited the specific one, but no success so far.
Unfortunately I don’t have a good understanding how information like policies are handled in istio. I’d expected pilot to distribute policies to the envoys, but I cannot nail it down in pilot’s logs, not even in log=tracing mode. Policies and destinationrules unfortunately don’t manifest in istioctl (or do they ?)

As an update: I created a meshpolicy with JWT auth and that works fine.

Second update: dumping the envoy config with http://localhost:15000/config_dump (after forwarding the port of the istio-proxy) shows clearly that the JWT config is available as http_filter

Martz · May 29, 2019, 9:59am

Could someone show the before/after to fix the Service Port Naming? What they changed, the Service? Virtual Service? Policy? Deployment? Gateway?

It’s really confusing.

Topic		Replies	Views
Struggling with JWT auth Security	1	695	January 4, 2022
Cors preflight does not work when Jwt Policy targets the Istio Ingress Gateway Security	5	4034	August 9, 2019
Istio JWT authentication does not seem to be working Security	7	5818	January 27, 2021
AuthorizationPolicy not Enforced Security security	2	950	October 8, 2020
Istio (1.6.2) : DENY policy in Authorization Policy does not work with Valid Token Security	1	459	July 10, 2020

JWT Policy does not take affect!

Related topics