Nginx Ingress Controller with Istio MTLS

1

Hi there,

I have a cluster that use Nginx Ingress and , and enabled auto MTLS for all services.

The internal services are all communicated fine with MTLS enabled and proper Peer Authentication policy applied, but i got an issue specifically for this communication link.

–> AWS ALB ----> Nginx Ingress Controller ----> Service

Namespaces

  1. default (injected with envoy sidecar). Applied with Peer Authentication Policy
  2. nginx-ingress (injected with envoy sidecar). No Policy applied

Updated with nginx-ingress-controller logs

2020/06/12 07:25:27 [error] 38#38: *5190 SSL_do_handshake() failed (SSL: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:SSL alert number 40) while SSL handshaking to 
upstream, client: 172.100.55.221, server: service.somewhere.com, request: "GET /admin/console.html 
HTTP/1.1", upstream: "https://10.100.xx.xx:443/admin/console.html", host: "service.somewhere.com"

When i remove the peer authentication policy for default namespace, the route works.

I think i do need some extra set-up or configuratio for non-istio ingress controller right? Appreciate if anyone can point me to a guide or reference :slight_smile:

2

Hi @robincher

see this task https://istio.io/latest/docs/tasks/traffic-management/ingress/kubernetes-ingress/ may this will help you.

3

Hi Shubham,

Thanks, i am actually trying to implement Nginx ingress instead.

So the ingress object i be configured with this

kubernetes.io/ingress.class: nginx

Is this possible ?

4

Hi

AFAIK this annotation is used to tell istio gateway controller that it should this ingress.(if you want to use k8s ingress with istio.)
means which controller handle this ingress. if you mention ingress then may be you can’t access the services in the sevices mesh.

https://istio.io/latest/docs/tasks/traffic-management/ingress/kubernetes-ingress/?_ga=2.211620759.36523360.1591586303-1565680694.1568085289#specifying-ingressclass (from Kubernetes 1.18, we can use ingress class instead of this annotation.)
https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-class

5

Yes correct, i am using Nginx as the ingress controller rather than Istio ingress controller.

My question is, how can i configure it such that Nginx controller such that,it can communicate with other pods using MTLS.

6

Don’t know exactly by writing Nginx it work with istio. for my understanding it should not work(not sure.).

see this for tls.

7

This might be what you have in mind. The same pattern will work for any service upstream from an NGINX Ingress Controller.

8

hi. if i have multiple hosts in my ingress how can this be done.

in our ingress we have multiple hosts specified ex:

spec:
rules:
- host: entry.host.test.com
   http:
    paths:
    - backend:
         serviceName: hi-service1-service
         servicePort: 8080
      path: /api/nam

    - backend:
        serviceName: hi-service2-service
        servicePort: 8080
      path: /api/tnt

- host: example1.com
   http:
    paths:
    - backend:
      serviceName: example1-service
      servicePort: 8080
      path: /

- host: example2.com
    http:
     paths:
     - backend:
         serviceName: example2-service
         servicePort: 8080
       path: /

for some reason traffic routing isn’t working as expected.
I read the docs from @Dave_Brewer

and added these annotations

 nginx.ingress.kubernetes.io/service-upstream: "true"
 nginx.ingress.kubernetes.io/upstream-vhost: mynamespace.example2-service.cluster.local

and traffic routing worked but the rest of the hosts are redirected to this service example2-service

any help is greatly appreciated :slight_smile:

1 Like
9

@robincher We are also trying the same thing,
GCP HTTPS LB —> Nginx Ingress Controller ----> Service

and getting 502 bad gateway error after adding Peer Authentication Policy.
can you pls help us?

10

I am facing the exact challenge. I have managed to inject Istio sidecars to Ingress NGINX pods, have used the upstream and vhost annotations to get the ingress-to-service traffic to flow through the mesh with mTLS, but for other services this won’t work as we can have only one such annotation. I thought I could use NGINX’s VirtualServer, as per this guide (I had a link here but this page does not allow me to post comment with links WTF?), but this seems to define one host as well.

I guess I could use multiple Ingress definitions, but this beats the whole purpose of using an Ingress.

Any solutions to this or ideas how to overcome that?