Fail to apply VirtualService and Gateway to Egress Gateway on port 80

egress-hook(2)
I am trying to redirect traffic destined for 10.15.12.31:31282 with the header “Host: foobar-2.acme.com” to the egress gateway in the same namespace, and have it exit the namespace through this egress gateway to the intended destination.

Our namespace looks as follows, sans the mTLS/OAuth. Everything is plain-text at the moment over HTTP.

For the first leg to hook the traffic over to the egress gateway I create the following VirtualService which applies to the mesh Gateway,

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-foobar-2-hook-to-egress-gateway
  namespace: fed-example-1
spec:
  gateways:
  - mesh
  hosts:
  - foobar-2.acme.com
  http:
  - match:
    - gateways:
      - mesh
      port: 31282
    route:
    - destination:
        host: istio-egressgateway.fed-example-1.svc.cluster.local
        port:
          number: 80
      weight: 100
    headers:
      request:
        set:
          host: "hook.foobar-2.acme.com"

This should hook all traffic from the workload with the host: foobar-2.acme.com on port 31282 over to the egress gateway on port 80 (since 80 is the only port exposed on the egress gateway service), while also manipulating the host header to hook.foobar-2.acme.com

This leg works fine where in I see the outbound log in the sidecar workload proxy,

[2019-07-18T19:51:20.933Z] "GET /worker/v1/meshExample/tracePath HTTP/1.1" 404 - "-" 0 0 2 2 "-" "curl/7.59.0" "65474f26-8f88-953e-9c4c-5e96661e5520" "hook.foobar-2.acme.com" "192.168.167.93:80" outbound|80||istio-egressgateway.fed-example-1.svc.cluster.local - 10.15.12.31:31282 192.168.167.83:49026 -

and also see it received on the gateway, albeit with a 404,

[2019-07-18T19:51:20.933Z] "GET /worker/v1/meshExample/tracePath HTTP/1.1" 404 - "-" 0 0 2 2 "-" "curl/7.59.0" "65474f26-8f88-953e-9c4c-5e96661e5520" "hook.foobar-2.acme.com" "192.168.167.93:80" outbound|80||istio-egressgateway.fed-example-1.svc.cluster.local - 10.15.12.31:31282 192.168.167.83:49026 -

In the second leg the following resources are setup to send the traffic out to the destined server at 10.15.12.31:31282 with host header Host: foobar-2.acme.com,
A Gateway resource for this egress gateway, that exposes port 80 for host hook.foobar-2.acme.com

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-egressgateway
  namespace: fed-example-1
spec:
  selector:
    istio: egressgateway
  servers:
  - hosts:
    - hook.foobar-2.acme.com
    port:
      name: http-hook-egress
      number: 80
      protocol: HTTP

A VirtualService associated with this Gateway to route the traffic to its intended destination route, while readjusting the header back to Host: foobar-2.acme.com

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-foobar-2-through-egress-gateway
  namespace: fed-example-1
spec:
  gateways:
  - istio-egressgateway 
  hosts:
  - hook.foobar-2.acme.com
  http:
  - match:
      port: 80
    route:
    - destination:
        host: foobar-2.acme.com
        port:
          number: 31282
    headers:
      request:
        set:
          host: "foobar-2.acme.com"

and a ServiceEntry for foobar-2.acme.com to be resolved to a STATIC address

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: foobar 
spec:
  hosts:
  - foobar-2.acme.com
  ports:
  - number: 31282
    name: http-port
    protocol: HTTP
  resolution: STATIC
  endpoints:
  - address: 10.15.12.31

However in this second leg the traffic intended for the listener “0.0.0.0_80” gets routed to the blackhole cluster.
I, for the life of me, cannot figure out why Pilot cannot find an associated Virtual host for the Gateway at port 80,
See logs,

2019-07-18T19:52:24.487893Z     info    ads     LDS: PUSH for node:istio-egressgateway-0-32-0-dbg-5f95d99bc4-czdvc.fed-example-1 addr:"192.168.167.93:49376" listeners:1 2430
2019-07-18T19:52:24.488031Z     warn    constructed http route config for port 80 with no vhosts; Setting up a default 404 vhost
2019-07-18T19:52:24.488127Z     info    ads     ADS: RDS: PUSH for node: istio-egressgateway-0-32-0-dbg-5f95d99bc4-czdvc.fed-example-1 addr:192.168.167.93:49376 routes:1
2019-07-18T19:52:24.492955Z     info    ads     CDS: PUSH 2019-07-18T19:52:24Z/502 for sidecar~192.168.167.74~goodbye-world-0-119-0-dbg-9855956bf-k25qt.fed-example-1~fed-example-1.svc.cluster.local-2988 "192.168.167.74:57298", Clusters: 179, Services 81
gc 5515 @70503.950s 0%: 0.039+71+1.5 ms clock, 0.15+2.6/14/0+6.3 ms cpu, 35->46->28 MB, 43 MB goal, 4 P

Any help here would be appreciated.

@Tao_Li ( Would really appreciate if you can CC anyone who might be able to help with this ? )

@howardjohn may know more

1 Like

Have you tried doing this using a subset, as shown in the egress gateway documentation?

I think you have to define a ServiceEntry for hook.foobar-2.acme.com as well. Pilot does not find such an entry and directs the traffic to blackhole cluster.

What would that ServiceEntry point to as the endpoint ? The egress service ?

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: amf-hook 
spec:
  hosts:
  - hook.amf-2.affirmed.com
  ports:
  - number: 80
    name: http-hook-port
    protocol: HTTP
  resolution: DNS 
  endpoints:
  - address: istio-egressgateway.fed-example-1.svc.cluster.local

So it seems to work if I set the Gateway to, foobar-2.acme.com instead of hook.foobar-2.acme.com.
Also works when the host is set to “*”, which I have switched to at the moment.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-egressgateway
  namespace: fed-example-1
spec:
  selector:
    istio: egressgateway
  servers:
  - hosts:
    - foobar-2.acme.com
    port:
      name: http-hook-egress
      number: 80
      protocol: HTTP

@nitishm I think you can use any endpoint, it does not matter, since the virtual service will reroute the traffic correctly, disregarding the endpoint.