Istio sidecar is unable to talk to Pilot. As a result we are unable to run our apps on Istio.
The logs from the istio-proxy container in the app pod are as below.
[Envoy (Epoch 0)] [2019-12-31 12:51:05.655][21][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:91] gRPC config stream closed: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination
2019-12-31T12:51:05.688008Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2019-12-31T12:51:07.688078Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2019-12-31T12:51:09.687973Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2019-12-31T12:51:11.688119Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2019-12-31T12:51:13.688051Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
The logs in the istio-proxy container in the Pilot pod shows some TLS handshake error.
[Envoy (Epoch 0)] [2019-12-31 11:43:25.464][17][debug][main] [external/envoy/source/server/server.cc:177] flushing stats
[Envoy (Epoch 0)] [2019-12-31 11:43:26.589][17][debug][upstream] [external/envoy/source/common/upstream/upstream_impl.cc:250] transport socket match, socket default selected for host with address 172.20.195.58:15019
[Envoy (Epoch 0)] [2019-12-31 11:43:28.603][35][debug][conn_handler] [external/envoy/source/server/connection_handler_impl.cc:333] [C5356] new connection
[Envoy (Epoch 0)] [2019-12-31 11:43:28.604][35][debug][connection] [external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:198] [C5356] handshake error: 1
[Envoy (Epoch 0)] [2019-12-31 11:43:28.604][35][debug][connection] [external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:226] [C5356] TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
[Envoy (Epoch 0)] [2019-12-31 11:43:28.604][35][debug][connection] [external/envoy/source/common/network/connection_impl.cc:193] [C5356] closing socket: 0
[Envoy (Epoch 0)] [2019-12-31 11:43:28.604][35][debug][conn_handler] [external/envoy/source/server/connection_handler_impl.cc:88] [C5356] adding to cleanup list
[Envoy (Epoch 0)] [2019-12-31 11:43:29.900][35][debug][conn_handler] [external/envoy/source/server/connection_handler_impl.cc:333] [C5357] new connection
[Envoy (Epoch 0)] [2019-12-31 11:43:29.901][35][debug][connection] [external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:198] [C5357] handshake error: 1
[Envoy (Epoch 0)] [2019-12-31 11:43:29.901][35][debug][connection] [external/envoy/source/extensions/transport_sockets/tls/ssl_socket.cc:226] [C5357] TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
We are using Istio 1.4.2. We are not using mTLS between our apps. We are not using istio-cni either. We checked all the bullet points mentioned in this doc.
Any help would be highly appreciated.