Egress Gateways - What's the Best Way to set up a "Core" Configuration, with Per Namespace Customisation?

Networking
1

Hi all, apologies for the confusing title.

I work for an institution that has a hard requirement for all cluster-external traffic to go through a controlled egress. The installation I’ve inherited has an egress gateway with SNI forwarding through port 433, with regular public chain TLS being forwarded through it. All of the egress related configuration exists in istio-system and is exported to all injected namespaces, as does the egress deployment itself.

The issue I am having is this: When creating new egress DestinationRules in an application team’s namespace to allow them to access additional external endpoints only the DestinationRules within that namespace are found for the egress host - so I lose the ability to connect to the “default” list. It appears that DestinationRules are merged for a host within a given namespace, but not across namespaces.

How can I arrange my configuration so that I can maintain a “base” configuration of endpoints that all workloads on the cluster can access, while giving individual namespaces additional endpoints?

As I see it there are a couple of solutions

  1. Create a second egress that can be configured on a per-namespace basis, keeping the “public” egress separate to the “per namespace” egress, and then defining Gateway, DestinationRule and VirtualService for that namespace within it

  2. Create some form of alias to the existing istio-egressgateway.istio-system.cluster.local service, so that when referenced by destination rules I don’t clobber the “default” rules imported from another namespace

  3. Continue to define the egress routing centrally, in istio-system, but restrict the routing to the endpoints to the namespaces that need them.

I’m not sure architecturally what the sensible thing to do here is - from looking at the description of the proxy configuration it seems like the DestinationHost is the “first order” object, and when defining egresses you’re always going to need to merge the rules so you end up with egress-host[1]:subsets[n]

Yours in confusion,
Tom.