Configurable trace header proposal

I’ve been frustrated about how Istio is forcing your applications to use the B3 headers, while Envoy proxy supports a lot more. So I decided to create a proposal… but as I’m quite new to the governance process I could use some help with a sponsor.

My high level idea is this:

  • Configure global what headers format should be used. This should be different for HTTP vs gRPC (gRPC has it own default binary tracing metadata).
  • Override the format per deployment, maybe your using an application from an other team that has an other propagation format.
  • Configurable format translation (example if you have REST<> gRPC configured: W3C Trace <-> gRPC binary translation)
  • This should be compatible with current implementations as you could still default to B3 if not configured.

This would enable my ideal setup:

  • Kubernetes, with the OpenCensus/OpenTelemetry agent/collector installed -> export to favorite tracing product
  • REST Container with OC/OT installed and the W3C headers configured
  • gRPC Container with OC/OT installed and using the default gRPC binary tracing
  • Istio+Application traces will be correlated, along with logs and metrics.

I could directly try to dive in the code, but I want to make the proper proposal. Comments and tips welcome. Thanks.

1 Like

@oaktowner looking for a sponsor on this topic :slight_smile:

I suggest attending a P&T WG meeting to discuss further, as an initial step.

Currently, Istio (via Envoy) provides support for Lightstep, Datadog, and Zipkin-compatible tracing. IIRC, Lightstep is using x-ot-span-context and Datadog uses a set of custom headers. There isn’t any strict requirement to use the B3 headers. It is just that the addon integrations that Istio provided installation support (running of demo service installs) for have been based on the Zipkin API.

With the extensibility v2 architecture (mixer-less Istio), tracing will be handled exclusively through Envoy proxies (and Envoy-based extensions). Even today, the recommendation is to use Envoy tracing integrations, only using Mixer-based tracing when absolute customization is required. At this point, Envoy tracing options will be the same as Istio tracing options.

Oh, thanks. I’ll dive first into the new architecture and see if I understand it enough. I’ve just started looking at the Envoy documentation and then you realize that indeed Envoy has lot more options.

Although it would not work with all tracers supported by Envoy, one solution would be to switch from native zipkin tracer to using the opencensus/zipkin exporter as default (or atleast as an option).

Then for all tracers supported by the opencensus config (currently zipkin, stackdriver and ocagent) it appears you could select between various propagation formats.

This would also support your requirement to use ot/oc collector - by using the ocagent exporter in Envoy.

1 Like

This is great info. I’ll certainly going to try this. This will be great material for a blog article :slight_smile: