Opencensus / Jaeger tracing disconnected from istio traces

Hi folks, this is somewhat of a follow-up to my issue here but reproduced in minikube so, hopefully, it’s a little easier to discuss exact setup details. You can find my project here with reproduction steps in the README. My problem is as follows. I have three services

  • obfuscator : a gRPC service that reverses whatever you send it
  • helloworld : an HTTP service that sends ‘hello, world’ to obfuscator and then wraps the result up in json
  • benchmarker : a service to generate load against helloworld

with the first two emitting opencensus traces directly to a central jaeger collector, the final relying entirely on the mesh traces for instrumentation. What I find after letting the cluster run for a while is that the appears to be a distinction between service names if the traces are istio sourced versus opencensus sourced. Unfortunately I am a new user and cannot post many images. You can find screenshots in my reproduction project under img/.

Opencensus originated traces are connected:

but istio originated traces do not appear to be.

Worth pointing out that the last two screenshots do show benchmarker.spangen collected in the trace. If you look at my jaeger configuration under k8s/manifests/jaeger/jaeger.yaml you’ll notice that it’s an in-memory setup but this issue is something I’ve managed to reproduce from my production cluster with disk-backed storage of traces. I have got istio pointed to jaeger like so and am working with istio 1.3.1, per the install script just linked.

Does anyone have a notion of what might be the matter?

I see you are using OpenCensus.OpenCensus uses for gRPC the build in binary trace headers (build into gRPC). When you configure OC to work with B3 headers this only will work on HTTP, not gRPC.

Thats why I propose making the headers format in Istio configurable, you can start following this thread:
Configurable trace header proposal

1 Like

This jives well with what I learned from the opencensus community: https://gitter.im/census-instrumentation/Lobby?at=5da4b28165dd8569a0d7da0e

Also related, now that I know what to look for: https://github.com/census-instrumentation/opencensus-go/issues/666

Yep saw that thread, but if your working in a poliglot environment you want it be configurable in Envoy.

Oh yes, absolutely agreed.