I am working on getting Istio deployed with the V2 telemetry pipeline installed and GKE Workload Identity. Things aren’t quite going as I’d like.
I got the mesh setup and it’s working great, but the Envoy proxies are all returning
CreateTimeSeries request failed (1 RPCs, 8 views, 8 timeseries): PERMISSION_DENIED: Permission monitoring.timeSeries.create denied (or the resource may not exist).
Sadly, I believe this is due to the fact that Workload Identity is turned on and consequently, the Envoy’s are attempting to use the (nonexistant) GSAs that would be bound to the KSAs. However, this poses a problem in my mind: I could add GSAs to all the pods, but the applications running in those pods don’t need the ability to write metrics to Cloud Monitoring, only the Envoys. Is there a way to configure Istio so that the Envoys can all use a single service account with the proper permissions to write to Cloud Monitoring?
Secret Discovery Service seems like a sane way to do this, but it seems like that’s really only used for TLS certs.