Istio 1.6.3: ingressgateway error "Proto constraint validation failed"

This is a new thread changing the title of the previous thread

I’m trying to setup this RequestAuthentication.

apiVersion: "security.istio.io/v1beta1"
kind: "RequestAuthentication"
metadata:
  name: "h-ingress-jwt"
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  jwtRules:
  - issuer: "http://192.168.202.21:8000/auth/realms/istio"
    jwksUri: "http://192.168.202.21:8000/auth/realms/istio/protocol/openid-connect/certs"

In the ingressgateway logs I see this error:

2020-07-09T09:36:50.669693Z     warn    Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 7 successful, 0 rejected; lds updates: 0 successful, 7 rejected
2020-07-09T09:36:52.669736Z     warn    Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 7 successful, 0 rejected; lds updates: 0 successful, 7 rejected
2020-07-09T09:36:54.670022Z     warn    Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 7 successful, 0 rejected; lds updates: 0 successful, 7 rejected
2020-07-09T09:36:55.068148Z     warning envoy config    [external/envoy/source/common/config/grpc_subscription_impl.cc:101] gRPC config for type.googleapis.com/envoy.api.v2.Listener rejected: Error adding/updating listener(s) 0.0.0.0_8443: Proto constraint validation failed (JwtAuthenticationValidationError.Providers[key]: ["embedded message failed validation"] | caused by JwtProviderValidationError.LocalJwks: ["embedded message failed validation"] | caused by DataSourceValidationError.InlineString: ["value length must be at least " '\x01' " bytes"]): providers {
  key: "origins-0"
  value {
    issuer: "http://192.168.202.21:8000/auth/realms/istio"
    local_jwks {
      inline_string: ""
    }
    payload_in_metadata: "http://192.168.202.21:8000/auth/realms/istio"
  }
}
rules {
  match {
    prefix: "/"
  }
  requires {
    requires_any {
      requirements {
        provider_name: "origins-0"
      }
      requirements {
        allow_missing {
        }
      }
    }
  }
}

0.0.0.0_8080: Proto constraint validation failed (JwtAuthenticationValidationError.Providers[key]: ["embedded message failed validation"] | caused by JwtProviderValidationError.LocalJwks: ["embedded message failed validation"] | caused by DataSourceValidationError.InlineString: ["value length must be at least " '\x01' " bytes"]): providers {
  key: "origins-0"
  value {
    issuer: "http://192.168.202.21:8000/auth/realms/istio"
    local_jwks {
      inline_string: ""
    }
    payload_in_metadata: "http://192.168.202.21:8000/auth/realms/istio"
  }
}
rules {
  match {
    prefix: "/"
  }
  requires {
    requires_any {
      requirements {
        provider_name: "origins-0"
      }
      requirements {
        allow_missing {
        }
      }
    }
  }
}


2020-07-09T09:36:55.706732Z     warning envoy config    [external/envoy/source/common/config/grpc_subscription_impl.cc:101] gRPC config for type.googleapis.com/envoy.api.v2.Listener rejected: Error adding/updating listener(s) 0.0.0.0_8443: Proto constraint validation failed (JwtAuthenticationValidationError.Providers[key]: ["embedded message failed validation"] | caused by JwtProviderValidationError.LocalJwks: ["embedded message failed validation"] | caused by DataSourceValidationError.InlineString: ["value length must be at least " '\x01' " bytes"]): providers {
  key: "origins-0"
  value {
    issuer: "http://192.168.202.21:8000/auth/realms/istio"
    local_jwks {
      inline_string: ""
    }
    payload_in_metadata: "http://192.168.202.21:8000/auth/realms/istio"
  }
}
rules {
  match {
    prefix: "/"
  }
  requires {
    requires_any {
      requirements {
        provider_name: "origins-0"
      }
      requirements {
        allow_missing {
        }
      }
    }
  }
}

0.0.0.0_8080: Proto constraint validation failed (JwtAuthenticationValidationError.Providers[key]: ["embedded message failed validation"] | caused by JwtProviderValidationError.LocalJwks: ["embedded message failed validation"] | caused by DataSourceValidationError.InlineString: ["value length must be at least " '\x01' " bytes"]): providers {
  key: "origins-0"
  value {
    issuer: "http://192.168.202.21:8000/auth/realms/istio"
    local_jwks {
      inline_string: ""
    }
    payload_in_metadata: "http://192.168.202.21:8000/auth/realms/istio"
  }
}
rules {
  match {
    prefix: "/"
  }
  requires {
    requires_any {
      requirements {
        provider_name: "origins-0"
      }
      requirements {
        allow_missing {
        }
      }
    }
  }
}


2020-07-09T09:36:56.669676Z     warn    Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 9 successful, 0 rejected; lds updates: 0 successful, 9 rejected

If I understand correctly, the ingressgatewaway didn’t receive a reply from http://192.168.202.21:8000/auth/realms/istio but ececuting a curl from the ingressgateway pod I receive some data in response:

kubectl exec -it istio-ingressgateway-66cc54b468-pkmd7 -n istio-system -- curl  http://192.168.202.21:8000/auth/realms/istio
{"realm":"istio","public_key":"MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwC48QVPBIDZ9ormmLqlwjZDCrur1HOyQOBsIRlWr6ieMQTNqqulWtRlmv+IcGScsXvZtotNfwJlRxGvYwoK/gg1J7gT4EzDwxVs7FKN7YktDKcT8HyYuAIXFZta15jxC+pOhJseZVQDpWPLjv5dn1vGEMe400v4uFTjqC1mZJohoPcNA5Q3IjwPC72LvJ0hGGPLzpv6gmMg/XNw3xK/z+5dctIH0PaOM+hh7wcDw45j32gEOyEHHxDWJzlJl1s4ckwVLCg+Wmof7UyW/aaqqoOqGxdjnNHz/nuYO61Rp7z1sHBn9/GMsJxZwlsxjCV23Vabccu1Iswjkn97sXZDtfQIDAQAB","token-service":"http://192.168.202.21:8000/auth/realms/istio/protocol/openid-connect","account-service":"http://192.168.202.21:8000/auth/realms/istio/account","tokens-not-before":0}

What I’m doing wrong?

it’s actually istiod to fetch the jwks and send it to Envoy, could you check if istiod is able to access http://192.168.202.21:8000/auth/realms/istio?

Sorry to be late.

Seems that istiod can access the issuer:

sysop@hdev:~$ kubectl exec -it istiod-777dc7dc48-rxf45 -n istio-system -- curl  http://192.168.202.21:8000/auth/realms/istio
{"realm":"istio","public_key":"MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwC48QVPBIDZ9ormmLqlwjZDCrur1HOyQOBsIRlWr6ieMQTNqqulWtRlmv+IcGScsXvZtotNfwJlRxGvYwoK/gg1J7gT4EzDwxVs7FKN7YktDKcT8HyYuAIXFZta15jxC+pOhJseZVQDpWPLjv5dn1vGEMe400v4uFTjqC1mZJohoPcNA5Q3IjwPC72LvJ0hGGPLzpv6gmMg/XNw3xK/z+5dctIH0PaOM+hh7wcDw45j32gEOyEHHxDWJzlJl1s4ckwVLCg+Wmof7UyW/aaqqoOqGxdjnNHz/nuYO61Rp7z1sHBn9/GMsJxZwlsxjCV23Vabccu1Iswjkn97sXZDtfQIDAQAB","token-service":"http://192.168.202.21:8000/auth/realms/istio/protocol/openid-connect","account-service":"http://192.168.202.21:8000/auth/realms/istio/account","tokens-not-before":0}sysop@hdev:~$ 

The complete istiod log is here

Opened also as a bug: 25578