Istio Ingress Gateway TCP TLS Termination

1

Hi,

My goal is to prove that Istio could work for my application deployment so I’ve started with a simple webapp and postgres server running in my cluster.

Both the webapp and postgres servers are unsecured (no TLS). I’ve been able to expose the ports both externally through the istio ingress gateway to allow access.

I’ve also been able to configure the istio ingress gateway to terminate HTTPS traffic for my webapp. I’m now trying to do the same for postgres.

I’m running into issues though when I try to connect:

./psql "port=31320 host=135.228.149.159 user=user dbname=url_shortener_db sslcert=/opt/ssl/certs/cert.pem sslkey=/opt/ssl/ssl.key sslmode=require sslrootcert=/opt/ssl/ca_cert.pem"
psql: received invalid response to SSL negotiation:

openssl s_client -connect 135.228.149.159:31320
Shows the certificates presented properly so I know the gateway has bound the port and is presenting the correct certificates.

I’m guessing I’ve got some config problem here but I’m not sure where.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: webapp-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 8080
      name: webapp-http
      protocol: HTTP
    hosts:
    - "*"
  - port:
      number: 9000
      name: webapp-https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - "*"
  - port:
      number: 5433
      name: postgres-tls
      protocol: TLS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - "*"
  - port:
      number: 5432
      name: postgres
      protocol: TCP
    hosts:
    - "*"

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: postgres
spec:
  hosts:
  - "*"
  gateways:
  - webapp-gateway
  tcp:
  - match:
    - port: 5432
    route:
    - destination:
        port:
          number: 5432
        host: postgres
  - match:
    - port: 5433
    route:
    - destination:
        port:
          number: 5432
        host: postgres

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: postgres-istio-mtls
spec:
  host: postgres
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL


istioctl version
client version: 1.4.0
citadel version: 1.4.0
egressgateway version: 1.3.2
galley version: 1.4.0
ingressgateway version: 1.4.0
pilot version: 1.4.0
policy version: 1.4.0
sidecar-injector version: 1.4.0
telemetry version: 1.4.0
data plane version: 1.4.0 (3 proxies), 1.3.2 (1 proxies)

kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:30:26Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

I’ve also tried enabling envoy proxy logging but I only seem to get messaging on the completion of a connection session so that’s not helpful in this case.

Any ideas or things I could try?

1 Like
2

@jonathw Were you able to solve this?

3

I think you need to change this to either SIMPLE or DISABLE as you resolve your certificates on your custom gateway.

4

So it turns out postgresql has a special negotiation it does to determine whether to use TLS or not. This would require work on ISTIO to support that protocol which as of the time of testing this it did not.

openssl for example (>1.1.1) had to add special support for this:

openssl s_client -starttls postgres -connect 1.1.1.1:5432 -showcerts

So in the end given that postgres is a big part of our current deployment we didn’t go with ISTIO. I ended up writing a custom TLS tool that acts as an init container to everything in the cluster and it cuts certs dynamically as things come up. This got us to the mutual auth we wanted.