Metric istio_request_messages_total isn't present


Here is my topology:

  • a grpc service outside the kubernetes cluster
  • an egress gateway that forwards traffic to it
  • a service that connect to this grpc service through the egress and receive messages from it

Everything is working fine, communication wise, the execution works but I don’t see the istio_request_messages_total metric in prometheus.

I can see in Kiali that the request are identified as grpc.
I can see requests going from the service and going to the egress gateway and then from the gateway to the external service in prometheus.

But still. The metric isnt’ here.

I tried to read istio’s source code to get a better sense of what’s going but could not pin down where that metric is filled.

My last resort would be to add custom metrics and scrape them from the listening services but I’d prefer to avoid that.

Thank you for your time and help