Istio-ingressgateway always returning 503s

Networking
1

I’ve been playing around with my Istio cluster configuration and I’ve ended up in a state I can’t debug my way out of.

I have the SDS+Gateway with a public IP configured. I have deployed the Istio HelloWorld app on port 5000. I can:

  • exec into istio-proxy on the helloworld-[rnd] pod and curl localhost:5000/hello - this works fine
  • check with istioctl proxy-config cluster (and such) from /docs/ops/troubleshooting/network-issues/ and /docs/ops/troubleshooting/proxy-cmd/ — all report OK to everything, SYNC:ed and such
  • I can do kubectl exec istio-ingressgateway-[rnd] /bin/bash and then curl helloworld.mynamespace:5000/hello successfully (it returns Hello version: v2 ...

But I can’t make the ingressgateway actually return anything but 503 when querying its publicly bound IP. If I query without the /hello path, it returns 404 instead, so it’s obviously trying to route to the helloworld service/deployment and failing.

So I’m in the state where I can actually contact my helloworld service from the Istio Ingress Gateway, when asking the gateway itself curl localhost/hello -i , or from ourside the network curl -i http://35.x.y.z/hello I always get 503 Service Unavailable Back

I don’t have any DestinationRule nor Policy applying to helloworld , and I have Istio in strict mTLS.

I could previously today access (other) services via the ingress gateway, but then I started cleaning things up (to the point when I only have the helloworld service VirtualService+Gateway and no others), and now it doesn’t work. It should be possible to debug.

What is wrong?

Not related (that I can tell):