Istiod: 15017 Connect: connection refused

Following the latest instruction https://github.com/istio/istio/issues/22944 I was able to get the telemetry working with GKE’s default istio setting enabled, and Private Cluster.

However, We got this recently after GKE recycled our nodes, the ports are open for 15017 and I can confirm that firewall is not blocking this.

failed calling webhook \\“validation.istio.io\\”: Post https://istiod.istio-system.svc:443/validate?timeout=30s: dial tcp 10.6.64.133:15017: connect: connection refused

Why was that? To solve this I have to

istioctl manifest apply --set components.telemetry.enabled=false
and
istioctl manifest apply --set components.telemetry.enabled=true

again to reinstall, and that seemed to have solved the issue. Any ideas why? This is quite serious as we can’t update Virtual Service at all.

Thanks

The Node evicted the Pod

Message:        The node was low on resource: ephemeral-storage. Container discovery was using 3112Ki, which exceeds its request of 0.