Distributed Tracing for non-HTTP protocols(or without HTTP headers)?

1

Istio distributed tracing requires http headers https://istio.io/docs/tasks/telemetry/distributed-tracing/overview/ . What is some of the services in the stack use a non-http protocol, like AMQP. How can we trace in such situations?

Thanks in advance.

2

Istio (and Envoy) only provide in-built support for tracing HTTP traffic currently - although Envoy does support other protocols, so hopefully in the future they will also be traced.

In the meantime, you will need to instrument your actual service to perform the tracing of non-HTTP endpoints - for example, if you are using Java and RabbitMQ, you could use: https://github.com/opentracing-contrib/java-spring-rabbitmq

3

Hello @objectiser , Thanks for the reply. I thought over your proposal and could not find an AMQP opentracing instrumentation for PYTHON.

Also, just confirming the optimality of approach, assuming the architecture shown below:

[(EP) Service A] —HTTP— > [(EP) Service B] — AMQP—> [(EP)Service C] —HTTP—> [(EP)Service D]

EP : Envoy (Istio) Proxy

EP of Service A,B and D will automatically send traces to jaeger backend using B3 headers but EP of service C wont be able to, since its an AMQP (Rabbit MQ) request. So, i need to manually inject span-context in request from service B to C and extract the span-context in service C and send it to jaeger backend manually, correct?

(I was hoping for a more cleaner/simpler solution, maybe a AMQP proxy for EP in Service C; rather than manually emitting traces)

These are the related resources i could find:
AMQP Tracer for python (could not find any):


How to inject an opentracing span into RabbitMQ: https://stackoverflow.com/questions/51423757/how-to-inject-an-opentracing-span-into-rabbitmq

Any thoughts, related resources (links for how to inject in AMQP , tracers ) might be helpful. Thanks.

4

Yes you are correct, there is no Python instrumentation for AMQP.

There was an issue in envoy to support AMQP https://github.com/envoyproxy/envoy/issues/3415 but went stale - it mentioned similarities to the kafka protocol support which is now available (I believe) - so potentially in the future it may get supported, but for now looks like your only option is to provide the instrumentation yourself (as you described) - and use the jaeger-client-python client to report the spans.

1 Like
5

@objectiser, Can you pl point me on how to manually send(report) a trace/span to jaeger backend ?? thanks.

6

Hopefully https://github.com/yurishkuro/opentracing-tutorial/tree/master/python will help.

1 Like