Istio without Gateway with nginx-ingress

1

Hello Everyone,

I use nginx as ingress and are not ready to leave nginx as our nginx does few conditional header manipulation before routing that is not possible with istio’s “virtualService”.

I want to use istio’s traffic routing features such as canary, mirroring, timeout and telemetery features such as prometheus, Jaeger and Graphana and may be few mixer policies but want nginx-ingress as entry point to the cluster and still able to take use of traffic routing.

Can someone guide me on how to achieve this ?

Reagards,
Sourabh

2

See GH Issue: looking for migration guide from nginx ingress to istio

When deploying nginx-ingress you can add the following pod annotation to tell istio to NOT intercept incoming traffic to nginx-ingress.

    traffic.sidecar.istio.io/includeInboundPorts: ""

Then for outbound traffic you can setup 2 annotations on your Ingress to avoid using endpoints directly and use istio instead.

    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/service-upstream: "true"
    nginx.ingress.kubernetes.io/upstream-vhost: <svc name>.<namespace>.svc.cluster.local
3

Thanks for your help Dev. I looked into that GH issue you mentioned. I still have few doubts.

First, I am not using “kind: Ingress” object. I am just using an nginx deployment that takes care of TLS termination and call routing.

Second, In “https://github.com/istio/istio/issues/7776#issuecomment-446847610”, He has mentioned he is not able to make use of “virtualservice” when using nginx ast the frontend but is able to make use of “virtualservice” when Gateway is friontend. I am facing the same problem.

If I have to use annotation is someway, can I use them even If I dont use “kind: ingress” object of kubernetes ?

4

Can someone help with this ?

I want to use nginx in front of pods which will have envoy in them. and i want to use istio’s traffic routing features using “virtualservices”.

I wont be using gateway at all.

5

Hi Sourabh,

I have a similar situation like yours – have nginx deployment with additional logic in it so do not want to get away from nginx at the moment.

He has mentioned he is not able to make use of “virtualservice” when using nginx ast the frontend but is able to make use of “virtualservice” when Gateway is friontend.

Did you find a solution/workaround for this? I’m yet to try the setup. By injecting envoy proxy in to nginx pod, wouldn’t nginx (proxy) requests be sent (routed through) to virtual services?

Thanks,
Satish

7

Anyone found a solution yet?

8

I actually got this to work, but in my case it only works if the Ingress definition that is given to the nginx-ingress server directs it to send traffic to services at port 80, specifically. Otherwise if I try to use the port that the K8s services are actually exposed through the virtual service does not apply. I’ve documented this setup in this github repo: GitHub - bob-walters/nginx-istio: A project to demonstrate using Istio traffic management for A/B service shift with an nginx-ingress, and the readme has the details of what I’ve tried.

I’m trying to determine at this point if the success I’ve been able to achieve is some kind of fluke that isn’t intended to work, or something that is only working due to the current behavior of retry logic. I.e. why it doesn’t work with other ports.

Also submitted github.com/istio/istio/issues/36705. One theory I have about why this is working is that the virtual service would work for the isito-ingressgateway, and so when the nginx-ingress service sends the traffic to its sidecar at port 80, it gets picked up and handled like istio ingress traffic.

9

I was able to get Virtual Services working with nginx-ingress (Istio 1.12). There were definitely several things that had to be perfectly lined up for this to work, and I’ve recorded the details and a repeatable experiment/setup on GitHub - bob-walters/nginx-istio: A project to demonstrate using Istio traffic management for A/B service shift with an nginx-ingress for anyone trying to deal with this in the future.