Certs termination on ALB. Is connection form ALB to gateway encripted?

Security
1

Hello,

Is connection between ALB and Gateway encrypted if TLS termination is done on ALB?

I am deploying Istio into AWS EKS (Istio 1.0.5, Kubernetes 1.11) and trying to follow advises in https://github.com/istio/istio/issues/6566

Confused by advise “PS: If you are doing SSL Termination with Amazon ELB, the protocol for port 443 should be HTTP, not HTTPS.”

It works and responds fine if I am using HTTP protocol in Gateway, following the advise:

  - port:
      number: 443
      name: https
      protocol: HTTP
    hosts:
    - test-service-*****.com

But it looks like message between ALB and Gateway is a plain http in this case. It looks confusing.
Is it a correct approach?

Certificate has been attached under my AWS load balancer

Annotations is added into istio-ingressgateway

annotations:
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:iam::****/my-    company-com-crt"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"

Thanks
Alex

2

If you terminate TLS on the ALB, then the ALB decrypts the incoming traffic and forwards plaintext to Istio gateway. If you want the traffic to stay encrypted all the way to the gateway, you should be able to turn off TLS termination.

What are you trying to accomplish?

3

One of use cases. For example, I have an ‘insecure’ application that was had no cert validation inside the pod.
In previous implementations (before Istio) we used TLS termination on AWS ELB. And insecure communication after ELB.

Now Istio provides much more secure option to use mTLS and in this case communication between gateway and pods are secured by sidecar.
And it looks that only piece of connection left not encrypted now is between ELB and istio gateway.
Am I correct?
What could be a reasonable approach?
Is it necessary to move the TLS termination to Gateway objects?

4

That makes sense.

One option would be to defer TLS termination in your ingress traffic to the Istio gateway. The Istio gateway can then re-encrypt traffic to the service backends using mTLS. Your ELB no longer terminates TLS and instead just acts as a load balancer / service VIP for the ingress.

5

Thank you.
We will try to relocate TLS termination into Gateway level.

Could you give an advice what LoadBalancer is more suitable for WebSocket applications in this architecture - CLB, NLB, ALB ?

6

@AlexD my ELB knowledge doesn’t go deep enough to make a recommendation for WebSocket applications. Maybe someone else in this community has advice?

7

Thank you @spikecurtis for your help!

8

@spikecurtis
Sorry to hijack a thread but i have a related question. Is it possible to terminate TLS at the ELB and also terminate traffic at the ingress gateway? I want to do this so I can reuse our AWS wildcard certs available on the elb and provision a self signed cert on the ingress gateway pods. I tried this and the only way I can get this to work is if I set the gateway hosts to a “*”. Once i specify a fully qualified domain name that would match both certs the connection never get’s through the gateway to the service pod. I just want to know if what I am trying is impossible and so i should give up. You seemed knowledgeable on this topic and thought i would ask.
Thanks. G

9

@Guido_Pepper what does it mean to “terminate TLS at the ELB and also terminate traffic at the ingress gateway”? Are you looking to terminate TLS in both places? You can only terminate the user’s TLS session once.

10

@spikecurtis thanks for taking time for this. Let me try to rephrase what I’m trying to accomplish.

When an external client on the internet connects to one of our services like coolsvc.cooldomain.net. We want the SSL cert provided back to the client to be our AWS wildcard cert that is configured in the elb asociated with the kubernetes service istio-ingressgateway in the istio-system namespace. The ELB in turn will create a connection with the istio-ingressgateway pod which is configured for SSL using a wildcard cert from our local CA.

So the clients on the internet get an SSL certificate from a well known CA but we also have encryption between the ELB and the istio-ingressgateway pod.

Using SSL at the ELB and in a pod works fine outside of istio. Trying to get this to work with the istio ingressgateway has been challenging.

From an client perspective the connection to the ELB works fine. But between the ELB and istio-ingressgateway pod something is happening. I’ve captured packets and can see the requests come into the instance but then a timeout occurs and curl exits with an HTTP 408.

I changed the gateway object for some testing and changed the hosts to “*” and then traffic started working. To me it feels like the traffic is getting into the ingress gateway and just not getting associated with the gateway. Does any of this make you think of something I have misconfigured or do you know the best way to enable some debug logging in the ingress-gateway pod to see what’s happening?

I have a pretty detailed post here How to terminate SSL at ELB and at istio ingressgateway that goes into details of the configurations of the various components.