I’m new to Istio in general, recently I took over a kubeflow project using istio on EKS.
I recently moved the istio deployments to a newly created node group (using NodeSelector in the deployment) to help protect it from being autoscaled. And now I’m having this weird problem.
At first, istio-ingressgateway had 0/1 pods, and investigation showed that it was happening because of the following error:
$ kubectl describe deploy istio-ingressgateway -n istio-system
Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedCreate 14m (x115 over 26h) replicaset-controller Error creating: Internal error occurred: failed calling webhook "sidecar-injector.istio.io": Post https://istio-sidecar-injector.istio.svc:443/inject?timeout=30s: service "istio-sidecar-injector" not found
But when I run
$ kubectl get service istio-sidecar-injector -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-sidecar-injector ClusterIP 10.100.85.254 <none> 443/TCP 66d
So the service appears to be there. I tried backing up, deleting and restoring the
istio-sidecar-injector deployment, and now also istio-sidecar-injector fails with the same error.
Any idea what can be causing this?
Thanks in advance