V2 Stackdriver Telemetry and Workload Identity

Policies and Telemetry
1

I am working on getting Istio deployed with the V2 telemetry pipeline installed and GKE Workload Identity. Things aren’t quite going as I’d like.

I got the mesh setup and it’s working great, but the Envoy proxies are all returning

CreateTimeSeries request failed (1 RPCs, 8 views, 8 timeseries): PERMISSION_DENIED: Permission monitoring.timeSeries.create denied (or the resource may not exist).

Sadly, I believe this is due to the fact that Workload Identity is turned on and consequently, the Envoy’s are attempting to use the (nonexistant) GSAs that would be bound to the KSAs. However, this poses a problem in my mind: I could add GSAs to all the pods, but the applications running in those pods don’t need the ability to write metrics to Cloud Monitoring, only the Envoys. Is there a way to configure Istio so that the Envoys can all use a single service account with the proper permissions to write to Cloud Monitoring?

Secret Discovery Service seems like a sane way to do this, but it seems like that’s really only used for TLS certs.

2 Likes
2

We are facing the same issue. Have you find a reasonable solution?

Thanks and best regards,
Fabian

3

Sadly, I have not. The project has been shelved for now so we’re looking at other stuff.

4

We’re also running in to this problem, looking for a nice solution :frowning:

5

https://github.com/istio/istio/issues/22658#issuecomment-662908816 has some related discussion about this.

6

I’m just hitting the same wall… Any progress in this matter?

7

You will need to patch the service account. I have done this in the operator configuration:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  profile: default
  components:
    pilot:
      enabled: true
      k8s:
        overlays:
        - kind: ServiceAccount
          name: istiod
          patches:
          - path: metadata.annotations.iam.gke.io/gcp-service-account
            value: ${service_account_email}

Also add the serviceAccount:${google_project_id}.svc.id.goog[istio-system/istiod] to roles/iam.workloadIdentityUser and then give ${service_account_email} the correct permissions. This should get rid of the errors.