Issue with federated scraping

Policies and Telemetry
1

Hi,
I have the following setup:

  • EKS Kubernetes cluster
  • Integrated Prometheus enabled
  • Deployed prometheus-operator in separate namespace
  • Implemented federation with ServiceMonitor as described at Best Practices.
    ** Currently not using recording rules.

Is it needed to add more matching expressions than the ones mentioned there?

    params:
      'match[]':
      - '{__name__=~"workload:(.*)"}'
      - '{__name__=~"pilot(.*)"}'

I cannot access all istio metrics, am I missing something?

The goal here is to store all Istio metrics in the external Prometheus with longer retention at production level.

Thanks for any help!

Regards
Bitemi

2

Hello,

I had to put the following in to get the dashboards to work:

params:
  'match[]':
  - '{__name__=~"workload:(.*)"}'
  - '{__name__=~"pilot(.*)"}'
  - '{__name__=~"envoy(.*)"}'
  - '{__name__=~"go(.*)"}'
  - '{__name__=~"istio(.*)"}'

I haven’t tested all the dashboards but there may be others. If you push custom metrics from those services you’ll need to capture those as well.

3

Thanks!
Do you use recording rules as well?