Kiali not showing traffic to all services

Policies and Telemetry Kiali
1

I have a service mesh with a handful of services. One service, reports-svc, is only accessible through another service auth-proxy. I have them both running, and the connection between them works, but Kiali is only showing traffic to auth-proxy.

Some details:

  • Istio 1.2.2
  • I have verified the sidecars are injected
  • I have verified both services are running
  • If I expose reports-svc via DNS I can see traffic to it
  • Service and VirtualService configs for both services are showing as ‘OK’ in Kiali
  • The service ports are named ‘http’ as described in the Istio docs

Anything else I should check?

Thanks for any advice you can offer!

2

Hello @davidham, could you check if Istio has collected telemetry for those services you don’t see ?
Basically, Kiali processes telemetry combined with Istio config + K8S objects to build all the representation.

My first check would be if for some reason, telemetry is empty.

Also, if you can add a way to reproduce it, we can take a look into details.

Thanks

3

Hi Lucas, thanks for the reply. Some additional details:

  • The inbound and outbound metrics for both services show the traffic, or they appear to, anyway.
  • The auth-proxy service is acting as a forward proxy, so it sends the request to reports-svc but does not handle the response. Would that affect how Kiali displays the two services?
  • We kubectl exec-ed into the auth-proxy container and the auth-proxy’s istio-proxy container, and curled reports-svc from there. Requests from auth-proxy show on the graph as originating from auth-proxy; ones from the istio-proxy show as coming from unknown. This kind of made sense.

Our best guess right now is its something in how auth-proxy forwards the request. Any thoughts?

Not sure how you might reproduce without the apps themselves, but I can post some of our K8s manifests if you like.

4

Hi, @davidham
Do you have access to Prometheus? I would like to know which time series are being stored.

5

I have noticed the missing edge when forwarding traffic use proxies. Though the prometheus has relevant metrics, kiali somehow does not show it

6

If Prometheus has telemetry, then in general Kiali should be showing it. Can you report this and provide all relevant details (replication procedures and definitely report what you see in prometheus - showing the prom telemetry helps alot in debugging things like this).