Some context:
We have an AWS EKS cluster, using the same VPC subnet as EC2 instances
In EC2, each component has it’s own security group, with default-deny on ingress
Now, we need to allow a workload in a pod access to a specific microservice running in EC2.
The trouble is, AWS doesn’t currently allow assigning a security group to a pod. So, our thought is to allow ingress from the EKS security group (that all worker nodes get assigned), and instead control access via egress restrictions in the istio cluster. Specifically, we’d force all egress traffic (http and https, but also tcp for e.g. RDS mysql/postgres) through an egress gateway, and control access via NetworkPolicy from the application pods to the egress pods. (side-note, I’m open to other options here if someone has come up with something better).
In researching, I found the article ‘Egress Gateways with TLS Origination’ (https://istio.io/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/) promising. This is close, but it breaks https queries from app pods (I created an issue to track the problem: https://github.com/istio/istio/issues/23740). I recognize that changing applications to use http is the likely response to this, but this is not an option for some sensitive microservices.
I’ve manipulated some settings from that example to try to get outbound https to route through the egress gateway, but haven’t had much success. Any guidance/advice appreciated.
Current config:
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: edition-cnn-com-egress
namespace: istio-system
spec:
host: istio-egressgateway
subsets:
- name: tls-origination
- name: tls-passthrough
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: edition-cnn-com-origination
namespace: istio-system
spec:
host: edition.cnn.com
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: SIMPLE
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: edition-cnn-com
namespace: istio-system
spec:
selector:
istio: egressgateway
servers:
- hosts:
- edition.cnn.com
port:
name: http-port-for-tls-origination
number: 80
protocol: HTTP
- hosts:
- edition.cnn.com
port:
name: tls-passthrough
number: 334
protocol: TLS
tls:
mode: PASSTHROUGH
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: edition-cnn-com
namespace: istio-system
spec:
hosts:
- edition.cnn.com
ports:
- name: http
number: 80
protocol: HTTP
- name: https
number: 443
protocol: HTTPS
resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: edition-cnn-com
namespace: istio-system
spec:
gateways:
- edition-cnn-com
- mesh
hosts:
- edition.cnn.com
http:
- match:
- gateways:
- mesh
port: 80
route:
- destination:
host: istio-egressgateway
port:
number: 80
subset: tls-origination
- match:
- gateways:
- edition-cnn-com
port: 80
route:
- destination:
host: edition.cnn.com
port:
number: 443
weight: 100
tls:
- match:
- gateways:
- mesh
port: 443
sniHosts:
- edition.cnn.com
route:
- destination:
host: istio-egressgateway
port:
number: 334
subset: tls-passthrough