Hi everyone,
We are building a setup where the egress gateway originates mTLS to a mesh-external host, roughly following this setup. However, in our setup the mesh-external host needs to identify the client using the client certificate. Hence, the egress gateway needs to send different client certificates based on the source of the traffic within the mesh.
Here’s our approach so far. We create a destination rule for our external host with two subsets, one for each client:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: originate-mtls-for-schlt-example
spec:
host: dev-s-this-02.draco-910.sccloudres.net
subsets:
- name: client-1
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: MUTUAL
clientCertificate: /etc/istio/dev-s-this-02-client-1-certs/tls.crt
privateKey: /etc/istio/dev-s-this-02-client-1-certs/tls.key
caCertificates: /etc/istio/dev-s-this-02-ca-certs/
sni: dev-s-this-02.draco-910.sccloudres.net
- name: client-2
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: MUTUAL
clientCertificate: /etc/istio/dev-s-this-02-client-2-certs/tls.crt
privateKey: /etc/istio/dev-s-this-02-client-2-certs/tls.key
caCertificates: /etc/istio/dev-s-this-02-ca-certs/
sni: dev-s-this-02.draco-910.sccloudres.net
We then create a virtual service which routes traffic to the external host dev-s-this-02.draco-910.sccloudres.net through the egress gateway. We detect which client is sending the request by matching on sourceLabels, in this test matching app: curl-1 or app: curl-2. Since we did not find a way to separate the traffic from each client across just one egress gateway, we created a second egress gateway service in istio. This allows us to then use separate routes and select the desired subset of the destination rule shown above.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: schlt-example-through-egress
spec:
hosts:
- dev-s-this-02.draco-910.sccloudres.net
gateways:
- schlt-example-egressgateway
- schlt-example-second-egressgateway
- mesh
http:
- match:
- sourceLabels:
app: curl-1
gateways:
- mesh
port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: schlt
port:
number: 443
weight: 100
- match:
- sourceLabels:
app: curl-2
gateways:
- mesh
port: 80
route:
- destination:
host: istio-second-egressgateway.istio-system.svc.cluster.local
subset: schlt
port:
number: 443
weight: 100
- match:
- gateways:
- schlt-example-egressgateway
port: 443
route:
- destination:
host: dev-s-this-02.draco-910.sccloudres.net
subset: client-1
port:
number: 443
weight: 100
- match:
- gateways:
- schlt-example-second-egressgateway
port: 443
route:
- destination:
host: dev-s-this-02.draco-910.sccloudres.net
subset: client-2
port:
number: 443
weight: 100
Our test curl client services are defined as follows. Note the app labels which we are setting.
apiVersion: v1
kind: Service
metadata:
name: curl-1
labels:
app: curl-1
spec:
ports:
- port: 80
name: http
selector:
app: curl-1
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: curl-1
spec:
replicas: 1
selector:
matchLabels:
app: curl-1
template:
metadata:
labels:
app: curl-1
spec:
containers:
- name: curl
image: cet-docker-virtual.artifactory.swisscom.com/appropriate/curl
command: ["/bin/sleep", "365d"]
imagePullPolicy: IfNotPresent
---
apiVersion: v1
kind: Service
metadata:
name: curl-2
labels:
app: curl-2
spec:
ports:
- port: 80
name: http
selector:
app: curl-2
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: curl-2
spec:
replicas: 1
selector:
matchLabels:
app: curl-2
template:
metadata:
labels:
app: curl-2
spec:
containers:
- name: curl
image: cet-docker-virtual.artifactory.swisscom.com/appropriate/curl
command: ["/bin/sleep", "365d"]
imagePullPolicy: IfNotPresent
Unfortunately, our setup does not work properly yet. In particular, the sourceLabels match does not seem to have an effect because all traffic from mesh seems to fall under the first match and consequently the first client certificate is always sent to the external host, irrespective of which client is calling.
Can anyone help us with the following questions:
- Does this setup make sense, or is there a completely different way of achieving the same goal?
- Are there any known limitations to matching
sourceLabelswhich could play a role here? - Do we really need to create a second egress gateway service just to separate the traffic or is there a way of marking traffic in one rule of a Virtual Service for processing in a second rule?
Any comments or hints appreciated! Let us know, if more details are needed.
Best regards and thanks!
Mathis
With Helm installation, you can enable SDS for egress. I don’t even know why are the gateways split into egress and ingress folders in istioctl. I hope there was a good reason for that.