In the gateways Helm chart, there is a ClusterRole and ClusterRoleBinding for the gateway service-account. In looking at the proxy-agent code under pilot, I couldn’t see where it makes any calls to the Kubernetes API server. For grins, I ran helm template using the sample values file for deploying a custom gateway and then removed all the ClusterRole and ClusterRoleBinding resources. The deployment worked just fine and the gateway came up without any errors. It connected to the mesh and external traffic is flowing nicely.
So why do we need ClusterRole and ClusterRoleBinding resources for the gateways child chart?
Here’s my scenario. I want to deploy an ingress-gateway in each namespace for a specific app with it’s own Service NodePort/LoadBalancer definition that’s separate from the main ingress-gateway in istio-system. I’ll label my custom ingress-gateway with my own label and use that label in my Gateway selector definitions specific for that app. With this config, I can control traffic into my app through its own gateway.
The problem I ran into is that the tiller I created in that namespace is granted admin access scoped to just that namespace, plus I create a special Role so it can also deploy Istio resources (e.g., VirtualService, Gateway, DestinationRule) in the namespace. So that means tiller can’t deploy cluster-wide resources. I do this to restrict the blast radius so devs working in one namespace can’t mess with tiller in another dev’s namespace or mess with the cluster.