How to originate TLS when forwarding to external ServiceEntry

[mike.martin.dev]

I am using Istio as an ingress controller within a kubernetes cluster. I have a single Gateway that receives HTTPS traffic for a host - my-service.domain.com

I need to split traffic for this host, based on URL prefix, between:

1. https : // my-service.domain .com/internal - a service internal to the cluster - accepts HTTP
2. https : // my-service.domain .com/external - an external service - accepts HTTPS

According to Istio / Egress TLS Origination I believe this should be possible.

I created a ServiceEntry for the external service and a DestinationRule also, but it seems that Istio is still sending HTTP traffic to the external service. (The external service indicates it is receiving HTTP traffic on port 443.)

How do I configure Istio so that it originates a new HTTPS connection when forwarding traffic to the external service?

Configs:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
spec:
  gateways:
    - ...
  hosts:
    - my-service.domain.com
  http:
    - match:
        - uri:
            prefix: /internal
      route:
        - destination:
            host: my-service.my-ns.svc.cluster.local
    - match:
        - uri:
            prefix: /external
      route:
        - destination:
            host: external-service.otherdomain.com

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
spec:
  hosts:
    - external-service.otherdomain.com
  ports:
    - number: 80
      name: http
      protocol: HTTP
    - number: 443
      name: https
      protocol: HTTPS
  location: MESH_EXTERNAL
  resolution: DNS

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: tls-foo
spec:
  host: external-service.otherdomain.com
  trafficPolicy:
    port:
      number: 80
    tls:
      mode: SIMPLE

[peterj]

I think you’re missing the targetPort in the Serviceentry:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
spec:
  hosts:
    - external-service.otherdomain.com
  ports:
    - number: 80
      name: http
      protocol: HTTP
      **targetPort: 443**
    - number: 443
      name: https
      protocol: HTTPS
  location: MESH_EXTERNAL
  resolution: DNS

(and you’re missing the portLevelSettings in the DestinationRule, but I think that was a typo on your side :))

[mike.martin.dev]

Thank you @peterj . I will try that.

What is the meaning of “targetPort” and how is that different than the main port number? What’s the difference between specifying a targetPort and not doing so?

The documentation just says “The port number on the endpoint where the traffic will be received” but that doesn’t tell me much.

[peterj]

When you direct the traffic in the VS you aren’t specifying port, and anything sent to external-service.otherdomain.com goes to the port 80 listener, even if you’re originating TLS (with DR). By adding the targetPort to the ServiceEntry your saying that anything sent to 80 should go to 443 (because you’re originating TLS with DR).

[mike.martin.dev]

But the problem seems to be that it is sending non-TLS traffic, not that it’s going to the wrong port.

I tried adding the targetPort but it made no difference. I’m still getting the same response from the external service:

The plain HTTP request was sent to HTTPS port

I also added a port number, as you said, to the VS, but that also made no difference. Here are my current configs:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
spec:
  hosts:
    - external-service.otherdomain.com
  ports:
    - number: 80
      name: http
      protocol: HTTP
      targetPort: 443
    - number: 443
      name: https
      protocol: HTTPS
  location: MESH_EXTERNAL
  resolution: DNS

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
spec:
  host: external-service.otherdomain.com
  trafficPolicy:
    portLevelSettings:
      - port:
          number: 80
        tls:
          mode: SIMPLE

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
spec:
  gateways:
    ...
  hosts:
    - my-service.domain.com
  http:
    - name: "internal-routes"
      match:
        - uri:
            prefix: "/internal"
      route:
        - destination:
            host: my-service.my-ns.svc.cluster.local
    - name: "external-routes"
      match:
        - uri:
            prefix: "/external"
      rewrite:
        authority: external-service.otherdomain.com
      route:
        - destination:
            host: external-service.otherdomain.com
            port:
              number: 443

[peterj]

Did you set the port to 443 in the virtualservice? Also are you terminating SSL at the gateway, or?

 route:
        - destination:
            host: external-service.otherdomain.com
            port:
                number:443

[mike.martin.dev]

I edited my last response above

[mike.martin.dev]

Yes, I want to terminate TLS at the gateway.

But then, after that, I want it to originate a new TLS connection when forwarding traffic to the external service, like it says here.

[peterj]

That’s odd; since the termination is at the gateway, you’re just originating a non-tls connection anyway. and your resources are exactly the same as with in the docs (granted, since you’re terminating at GW, you can send the traffic in VS to 80, so it gets originated with DR).

I am setting up a test cluster to try this out - my guess is the docs should be correct, but you never know

[peterj]

Here’s what I tried and it worked – note that I am sending traffic to GW on 80, but I don’t think it should matter if it’s 443 because it should be terminated (I am assuming you have tls mode SIMPLE on the GW + credentials with the cert).

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: hello-com-to-cnn
spec:
  gateways:
  - gateway
  hosts:
  - hello.com
  http:
    - match:
        - uri:
            prefix: "/external"
      rewrite:
        authority: edition.cnn.com
        uri: /
      route:
        - destination:
            host: edition.cnn.com
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: cnn
spec:
  hosts:
  - edition.cnn.com
  location: MESH_EXTERNAL
  resolution: DNS
  ports:
  - number: 80
    name: http
    protocol: HTTP
    targetPort: 443
  - number: 443
    name: https
    protocol: HTTPS
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - hello.com
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: edition-cnn-com
spec:
  host: edition.cnn.com
  trafficPolicy:
    portLevelSettings:
      - port:
          number: 80
        tls:
          mode: SIMPLE

And then:

$ curl -H "Host: hello.com" -sSL -o /dev/null -D - http://<ingress-gw-ip>/external

HTTP/1.1 200 OK

[mike.martin.dev]

Can you tell me what’s different from your config and mine?

[mike.martin.dev]

I tried the configs you gave, but now I am getting:

HTTP/2 502
upstream connect error or disconnect/reset before headers. reset reason: protocol error

(I suspect this has the same meaning as the other error I was getting. It’s just that edition.cnn.com terminates the connection immediately, whereas the other external service I’ve been trying sends back an actual response body with a more specific error.)

[mike.martin.dev]

Is there a restriction as to what configs Istio is able to apply, based on the k8s namespace?

The gateway is not under my control and is in a different namespace than my other configs.

My VirtualService has a spec.gateways that matches the Gateway, but I’m wondering whether I need to add a selector to the other configs too (DR, SE)?

[mike.martin.dev]

What appears to be happening to me is that the DestinationRule is being ignored. Whether I add or remove it, I get the same results.

[peterj]

By default, all Istio resources are available to use across the namespaces-- unless you have an exportTo field explicitly specified (that would make the resource available in specific namespaces only).