Jaeger Tracing application traces aren't seen

1

Hi All,

We are enabling jaeger tracing and we were trying to bring up below example.

We were able to bringup but when we trace we see only default traces and no application traces… Any help on the same ?.

Screenshot 2020-11-10 164531
Screenshot 2020-11-10 1645311192×573 63.1 KB

rgds
Balaji

2

The application should be injecting a span using the x-b3-* headers generated, this would be then seen as a child span.

3

Hi @kunalekawde ,
Perhaps, make sure that you have a correct configuration for your Jaeger collector, double-check that the endpoint defined in your app (generally as the environment variable JAEGER_ENDPOINT) matches the exposed Jaeger collector service. Also I remember that the Jaeger collector service was missing in some versions of Istio (demo profile) and it was necessary to create it manually, but this should be fixed now I think.

4

Thanks kunalekawde
I see its sending fine. Please find below logs and snapshot have attached gthub path of the example I am testing.

Headers: [host:“service-b:8081”, accept:“text/plain, application/json, application/*+json, /”, baggage-my-baggage:“CarlosBalaji”, user-agent:“Apache-HttpClient/4.5.9 (Java/1.8.0_261)”, accept-encoding:“gzip,deflate”, x-forwarded-proto:“http”, x-request-id:“74d10067-5224-9d1a-92e6-d37a302fcaa5”, x-envoy-attempt-count:“1”, content-length:“0”, x-forwarded-client-cert:“By=spiffe://cluster.local/ns/default/sa/default;Hash=ffdb0b2db4e751b7b9f6ca6e6c44517191e412e6383062f72e28a25291a6b4b7;Subject=”";URI=spiffe://cluster.local/ns/default/sa/default", x-b3-traceid:“c77dbe49681ab16b4ea0a38ddbbad513”, x-b3-spanid:“f6e926d14b7a3a08”, x-b3-parentspanid:“d25ae10483dac5e1”, x-b3-sampled:“1”]
2020-11-12 09:48:28.403 INFO 6 — [nio-8081-exec-3] i.j.internal.reporters.LoggingReporter : Span reported: c77dbe49681ab16b4ea0a38ddbbad513:68430d2375cda1ac:f6e926d14b7a3a08:1 - formatGreeting

Above is the logs

Screenshot 2020-11-12 152016
Screenshot 2020-11-12 1520161913×620 155 KB

And in the screen shot I see some skewed warning.

@jotak I am using below example.

https://github.com/ibm-cloud-architecture/learning-distributed-tracing-101/tree/master/lab-jaeger-istio-java
is from where I have taken the project and tried.

rgds
Balaji

5

invalid parent span is because jaeger is not able to co-relate this span with parent span , so parent it has not received yet or some other case of missing. Check the jaeger agent config IP configured. Ideally applications would be sending to agent.

6

Thanks a lot @kunalekawde , that helped me to debug and got it working. Apparently examples are written with assumption that production template is being used for jaeger, but apparently istio site is specifying a non production url. Hence any example we use need to make sure we are doing as below.

Says

$ kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.7/samples/addons/jaeger.yaml

Once we do as above then we should make sure our endpoint is aligned to this local all-in-one jaeger

kubectl get pods -o wide --all-namespaces  | grep jaeger
istio-system   jaeger-5795c4cf99-hwj5m                    1/1     Running            0          28m     10.244.25.54    slave    <none>           <none>

after getting this IP need to change in your deployment yaml as

--- a/lab-jaeger-istio-java/istio-java.yaml
+++ b/lab-jaeger-istio-java/istio-java.yaml
@@ -34,11 +34,11 @@ spec:
     spec:
       containers:
         - name: app
-          image: csantanapr/service-a-java-istio
+          image: xxxxx/service-a-java-istio
           #image: image-registry.openshift-image-registry.svc:5000/default/service-a-java-istio
           env:
             - name: JAEGER_ENDPOINT
-              value: http://jaeger-collector.istio-system.svc:14268/api/traces
+              value: http://10.244.25.54:14268/api/traces
             - name: JAEGER_PROPAGATION
               value: b3
             - name: SERVICE_FORMATTER