Hi folks,
We’ve updated our Lightstep config for our mesh to consist of the following profile settings:
meshConfig:
defaultConfig:
tracing:
tlsSettings:
mode: "SIMPLE"
caCertificates: "/etc/lightstep/cacert.pem"
values:
global:
tracer:
lightstep:
address: [redacted]:9292
accessToken: [access token]
proxy:
tracer: lightstep
pilot:
traceSampling: 100
… but our services report missing traces at the mesh level. This only occurred with the upgrade / change in config to using this meshConfig field. Our other cluster on Istio 1.5.1 reports Lightstep traces for the same calls just fine.
There’s a lot of this in the ingress-gateway pod logs, but no issues in the application pod logs . We find this unusual because the ingress-gateway typically is unaware of the Lightstep configs.
2021-03-03T18:30:53.679547Z | error | sds | resource:file-root:/etc/lightstep/cacert.pem Close connection. Failed to get secret for proxy “router~10.1.10.164~istio-ingressgateway-5b7475bbb6-547xf.istio-system~istio-system.svc.cluster.local” from secret cache: open /etc/lightstep/cacert.pem: no such file or directory |
---|---|---|---|
2021-03-03T18:30:53.679632Z | info | sds | resource:file-root:/etc/lightstep/cacert.pem connection is terminated: rpc error: code = Canceled desc = context canceled |
2021-03-03T18:30:53.679801Z | warning | envoy config | StreamSecrets gRPC config stream closed: 2, open /etc/lightstep/cacert.pem: no such file or directory |
2021-03-03T18:31:16.104033Z | info | sds | resource:file-root:/etc/lightstep/cacert.pem new connection |
Has anybody encountered this before? This is a big blocker for us in this upgrade.