Kiali not showing traffic to all services

I have a service mesh with a handful of services. One service, reports-svc, is only accessible through another service auth-proxy. I have them both running, and the connection between them works, but Kiali is only showing traffic to auth-proxy.

Some details:

  • Istio 1.2.2
  • I have verified the sidecars are injected
  • I have verified both services are running
  • If I expose reports-svc via DNS I can see traffic to it
  • Service and VirtualService configs for both services are showing as ‘OK’ in Kiali
  • The service ports are named ‘http’ as described in the Istio docs

Anything else I should check?

Thanks for any advice you can offer!

Hello @davidham, could you check if Istio has collected telemetry for those services you don’t see ?
Basically, Kiali processes telemetry combined with Istio config + K8S objects to build all the representation.

My first check would be if for some reason, telemetry is empty.

Also, if you can add a way to reproduce it, we can take a look into details.


Hi Lucas, thanks for the reply. Some additional details:

  • The inbound and outbound metrics for both services show the traffic, or they appear to, anyway.
  • The auth-proxy service is acting as a forward proxy, so it sends the request to reports-svc but does not handle the response. Would that affect how Kiali displays the two services?
  • We kubectl exec-ed into the auth-proxy container and the auth-proxy’s istio-proxy container, and curled reports-svc from there. Requests from auth-proxy show on the graph as originating from auth-proxy; ones from the istio-proxy show as coming from unknown. This kind of made sense.

Our best guess right now is its something in how auth-proxy forwards the request. Any thoughts?

Not sure how you might reproduce without the apps themselves, but I can post some of our K8s manifests if you like.

Hi, @davidham
Do you have access to Prometheus? I would like to know which time series are being stored.

I have noticed the missing edge when forwarding traffic use proxies. Though the prometheus has relevant metrics, kiali somehow does not show it

If Prometheus has telemetry, then in general Kiali should be showing it. Can you report this and provide all relevant details (replication procedures and definitely report what you see in prometheus - showing the prom telemetry helps alot in debugging things like this).