I would like to get feedback from the community about the way that you leverage the distributed tracing in Istio to find issues in your web applications. Let me elaborate on this regard.
Currently, when you install Istio you get default distributed tracing for free where you can display the Jaeger’s UI and see “stuff” going on. However, I’m more than skeptical on whether that provides actual value, mainly due to the fact that there is a huge amount of noise introduced by traces and spans coming and going from the Istio’s control plane components, i.e. mixer, policy, ingress, etc.
Let me share a stupid example about what I mean here. I have a simple application consisting of a UI that basically calls a backend to retrieve information (data indexed in Elasticsearch) and displays it to the user in a grid way. After using the application for a while, I go to Jaeger and when I click into a trace, I get 10 spans out of which only 2 of them provide real value in terms of the business transaction I’ve made (and even 1 of those 2 is actually the internal call of the sidecar):
Therefore making it quite hard to see clearly what’s going on.
Is anyone facing this kind of thought? I would love to hear how you out there are really benefiting from all this magic provided by Istio.