Upgrade to 1.5.4 and ingressgateway

Hello,

Did test upgrade from 1.4.7. to 1.5.4 (helm). Noticed that istio-ingressgateway pods sometimes get event for failing readiness probe:

“Warning Unhealthy 2s (x3 over 6s) kubelet, gke-dev-cookie-platform-node-pool-d383a6b3-8a61 Readiness probe failed: Get http://xxx:15020/healthz/ready: dial tcp xxx:15020: connect: connection refused”

Eventually container is restarted. It does not happen all the time. Not for all pods(ingressgateway). I did some investigation and it looks it is somehow connected to hpa.

Question is it expected? So I do not have to worry. Or should I investigate more(but what)?

Logs from pod:

istio-proxy [Envoy (Epoch 0)] [2020-06-25 10:48:14.080][18][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:91] gRPC config stream closed: 14, no healthy upstream
istio-proxy [Envoy (Epoch 0)] [2020-06-25 10:48:14.080][18][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:54] Unable to establish new stream
istio-proxy [Envoy (Epoch 0)] [2020-06-25 10:53:22.624][31][warning][filter] [src/istio/mixerclient/report_batch.cc:110] Mixer Report failed with: UNAVAILABLE:upstream connect error or disconnect/reset before headers. reset reason: connection termination
istio-proxy [Envoy (Epoch 0)] [2020-06-25 11:03:52.199][18][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:91] gRPC config stream closed: 13,
istio-proxy [Envoy (Epoch 0)] [2020-06-25 11:11:37.294][18][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:91] gRPC config stream closed: 13,
istio-proxy [Envoy (Epoch 0)] [2020-06-25 11:22:21.157][18][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:91] gRPC config stream closed: 13,
1 Like