Our EKS backplane is getting to small for our cluster setup, and while we wait for AWS to upgrade us we are facing a couple of issues… Sometimes the public services that goes through istios ingress gateway is experiencing 503 errors.
In the logs from the gateway pod I see k8s api timeout but no logs saying that it can’t serve requests.
In istiod I see controller election errors.
2020-11-02T13:59:04.212123Z error error retrieving resource lock istio-system/istio-namespace-controller-election: Get “https://10.100.0.1:443/api/v1/namespaces/istio-system/configmaps/istio-namespace-controller-election”: dial tcp 10.100.0.1:443: i/o timeout
Could the api timeouts cause istio to return 503’s or should we look elsewhere?
/Ole