Waiting for file

Hey guys,

So, I installed istio by running:

istioctl manifest apply --set profile=minimal,gateways.enabled=true
(Well, I actually did “generate”, then updated the yaml before applying it.)

because I only need the Ingress gateway for now. I was hoping to test it without TLS first, i.e. access my services using http, which I was able to do with the Quick Install.

However, the two pods (pilot and ingressgateway) are failing to start up. I see the following in their logs (istio_proxy container’s in case of the pilot pod):

|2019-11-26T01:44:59.435745Z|info|Monitored certs: string{"/etc/certs/cert-chain.pem", “/etc/certs/key.pem”, “/etc/certs/root-cert.pem”}|
|2019-11-26T01:44:59.435751Z|info|waiting 2m0s for /etc/certs/cert-chain.pem|
|2019-11-26T01:45:00.437327Z|info|waiting for file|
|2019-11-26T01:45:00.537534Z|info|waiting for file|
… it goes on and on…

Is there a way for them to stop looking for these files? Or should I just go ahead and embrace TLS from the get go? I’m assuming this is TLS-related.

Thanks,
J

I got past these for both pilot and ingressgateway by running the following:

kubectl create secret generic istio.istio-ingressgateway-service-account -n istio-system
–from-file=samples/certs/key.pem --from-file=samples/certs/root-cert.pem
–from-file=samples/certs/cert-chain.pem

kubectl create secret generic istio.istio-pilot-service-account -n istio-system
–from-file=samples/certs/key.pem --from-file=samples/certs/root-cert.pem
–from-file=samples/certs/cert-chain.pem

Prior to that, I had to copy ca-key.pem into key.pem as the latter did not exist.

Also, when I ran them initially, they didn’t take effect. I had to uninstall Istio, create the namespace “istio-system”, run the above commands, and re-install Istio using the manifest command.

Is this anywhere in the documentation? Are they supposed to part of the installation steps? I’m still unsure as to why no one else seems to be running into the same issue. Note that also I tried generating using the default profile, thinking that maybe the minimal profile was missing something important. But alas, I ran into the same issue.

Anyway, so I got past that. However, my pods are still unhealthy.

The istio-proxy container in the pilot pod seems to be okay now… I see a bunch of lines similar to the following:

[2019-11-27T23:08:48.982Z] “POST /istio.mcp.v1alpha1.ResourceSource/EstablishResourceStream HTTP/2” 200 UH 0 0 0 - “-” “grpc-go/1.24.0” “aa37a9c9-af4d-4582-9063-c775b262ed3d” “localhost:15019” “-”

The discovery container, OTOH, is giving me the errors at the bottom of this post (It’s too long so I’m pasting it last).

The ingressgateway is giving me:

2019-11-27T23:12:04.605121Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected

Any ideas?

Here are the logs from the discovery container in the pilot pod:

2019-11-27T23:09:46.033256Z info mcp (re)trying to establish new MCP sink stream

2019-11-27T23:09:46.033349Z info mcp New MCP sink stream created

2019-11-27T23:09:46.033834Z error mcp Error receiving MCP resource: rpc error: code = Unavailable desc = no healthy upstream

2019-11-27T23:09:46.033856Z error mcp Error receiving MCP response: rpc error: code = Unavailable desc = no healthy upstream

2019-11-27T23:09:46.048614Z info Configuration not synced: first push for [istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies] not received

2019-11-27T23:09:46.148573Z info Configuration not synced: first push for [istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs] not received

2019-11-27T23:09:46.248600Z info Configuration not synced: first push for [istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings] not received

2019-11-27T23:09:46.348600Z info Configuration not synced: first push for [istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs] not received

2019-11-27T23:09:46.448607Z info Configuration not synced: first push for [istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs] not received

2019-11-27T23:09:46.548634Z info Configuration not synced: first push for [istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs] not received

2019-11-27T23:09:46.648575Z info Configuration not synced: first push for [istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/destinationrules istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/serviceentries istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/virtualservices istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/envoyfilters] not received

2019-11-27T23:09:46.748657Z info Configuration not synced: first push for [istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs] not received

2019-11-27T23:09:46.848659Z info Configuration not synced: first push for [istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies] not received

2019-11-27T23:09:46.948649Z info Configuration not synced: first push for [istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/serviceentries istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/virtualservices istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/envoyfilters istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/destinationrules] not received

2019-11-27T23:09:47.033995Z info mcp (re)trying to establish new MCP sink stream

2019-11-27T23:09:47.034086Z info mcp New MCP sink stream created

2019-11-27T23:09:47.034603Z error mcp Error receiving MCP resource: rpc error: code = Unavailable desc = no healthy upstream

2019-11-27T23:09:47.034623Z error mcp Error receiving MCP response: rpc error: code = Unavailable desc = no healthy upstream

2019-11-27T23:09:47.048600Z info Configuration not synced: first push for [istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles] not received

2019-11-27T23:09:47.149509Z info Configuration not synced: first push for [istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways] not received

2019-11-27T23:09:47.248613Z info Configuration not synced: first push for [istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies] not received

2019-11-27T23:09:47.348581Z info Configuration not synced: first push for [istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs] not received

2019-11-27T23:09:47.448640Z info Configuration not synced: first push for [istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs] not received

2019-11-27T23:09:47.548603Z info Configuration not synced: first push for [istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies] not received

2019-11-27T23:09:47.648580Z info Configuration not synced: first push for [istio/networking/v1alpha3/destinationrules istio/networking/v1alpha3/sidecars istio/mixer/v1/config/client/quotaspecs istio/rbac/v1alpha1/clusterrbacconfigs istio/networking/v1alpha3/serviceentries istio/config/v1alpha2/httpapispecs istio/config/v1alpha2/httpapispecbindings istio/authentication/v1alpha1/meshpolicies istio/rbac/v1alpha1/serviceroles istio/networking/v1alpha3/virtualservices istio/networking/v1alpha3/gateways istio/authentication/v1alpha1/policies istio/rbac/v1alpha1/rbacconfigs istio/networking/v1alpha3/envoyfilters istio/mixer/v1/config/client/quotaspecbindings istio/rbac/v1alpha1/servicerolebindings istio/security/v1beta1/authorizationpolicies] not received