Can't get Ingress gateway to start up when SDS is enabled

Hi folks,

I’ve tried this earlier and got the same result. I’m trying it again.

I enabled SDS for the Ingress gateway.

  1. istio-security-post-install-1.4.2 shows as 1/2 Running. Istio-init and kubectl completed. Istio-proxy is running. Is this normal? I’ve always thought that it was one of those install-time pods that would complete and would no longer be needed afterwards. So I’m confused as to why it’s still running. What is its purpose?

  2. The more important thing in this post:

istio-ingressgateway-794d7fc889-4mzs5 1/2 CreateContainerError 0 39m

It is ingress-sds that fails. But when I do “describe” or “logs”, that’s also all I see… “CreateContainerError”. How else can I troubleshoot this?

Here are the relevant sections in the “describe” output:

    Container ID:   
    Image ID:       
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CreateContainerError
    Ready:          False
    Restart Count:  0
      cpu:     2
      memory:  1Gi
      cpu:     100m
      memory:  128Mi
      ENABLE_WORKLOAD_SDS:         false
      INGRESS_GATEWAY_NAMESPACE:   istio-system (v1:metadata.namespace)
      /var/run/ingress_gateway from ingressgatewaysdsudspath (rw)
      /var/run/secrets/ from istio-ingressgateway-service-account-token-btk2r (ro)

  Type     Reason            Age                  From                               Message
  ----     ------            ----                 ----                               -------
  Normal   Scheduled         40m                  default-scheduler                  Successfully assigned istio-system/istio-ingressgateway-794d7fc889-4mzs5 to
  Normal   Pulling           40m                  kubelet,  Pulling image ""
  Normal   Pulled            40m                  kubelet,   Successfully pulled image ""
  Normal   Pulled            40m                  kubelet,   Container image "" already present on machine
  Normal   Created           40m                  kubelet,   Created container istio-proxy
  Normal   Started           40m                  kubelet,   Started container istio-proxy
  Warning  Unhealthy         40m (x6 over 40m)    kubelet,   Readiness probe failed: Get dial tcp connect: connection refused
  Warning  Failed            40m (x4 over 40m)    kubelet,   Error: Error response from daemon: No command specified
  Normal   Pulled            40m (x3 over 40m)    kubelet,   Container image "" already present on machine
  Warning  Unhealthy         40m (x2 over 40m)    kubelet,  Readiness probe failed: HTTP probe failed with statuscode: 503
  Warning  DNSConfigForming  39s (x189 over 40m)  kubelet,   Search Line limits were exceeded, some search paths have been omitted, the applied search line is: istio-system.svc.cluster.local svc.cluster.local cluster.local

This is all I see in the ingress-sds logs:

Error from server (BadRequest): container "ingress-sds" in pod "istio-ingressgateway-794d7fc889-4mzs5" is waiting to start: CreateContainerError

Anything obvious to you, folks?

Anywhere else I can look to get to the bottom of the failure?


Problem solved. Turns out I had a bad (but non-corrupt) copy of the node-agent-k8s image. It was missing the Entrypoint. I remember downloading it from the prerelease directory. I pulled a new one and it worked. And it’s no April Fools’ joke.