Duplicate totals in istio_requests_total metric - labels populated in 1 total and Unknown in the duplicate

Policies and Telemetry
1

Hi.

I have a prometheus scrape job configured to scrape our Istio Ingress Gateway pods. When I look at the metrics, istio_requests_total for example, I can see totals returned where the destination_* labels are populated correctly and then another set of totals where the labels are all set to unknown
Has anyone else had this issue?

We are running Istio v1.15.1 and we use it just as an Ingress solution for now.

Thanks

2

perhaps you could restrict your query to reporter=“destination”

3

The reporter is always source as the metrics are coming from the Ingress Gateway pods.

It seems that the duplicate totals only affect non-200 response codes which is a bit odd.

4

Oh right, that sense that it’s just source reporting. Remember that a failed request may not make it to a destination workload, and as such there will be no destination information.

5

Thanks. The number of response codes I see would indicate it for to the destination but I will double check.