We activated istio in our lab in PERMISSIVE mode and a few services broke. One of them is a loadbalancer for mariadb servers that live outside of kubernetes. The loadbalancer live in pods and the service fronting them is using a LoadbalancerIP obtained through MetalLB.
In schema that follow, the Mariadb servers live in vms. Before injection with with the side car, everything worked just fine. Connexion path 1,2, 3 and 4 worked juste fine.
Once side-car injection has happened, a few connexions stoped working.
That is, connexion 1 and 3 that try to talk to the LoadBalancerIP (which is obviously not a clusterIP) do not work. It obviously has to do with the server side protocol that is mysql. I have modified the service in the
infra-lb namespace to tag correctly the service with either or both
appProtocol field or to make sure the name respects
name: <protocol>[-<suffix>] syntax.
To no avail. The behaviour is still different between the two IPs. By the ways, CentOS-tools is simply a centos image with lots of tools that we have in each of our kubernetes clusters so we can comfortably tests things.
Notice that the CentOS-tools pod can talk directly to mariadb through connexion without any issues.
I can fix the problem for the CentOS tools image which belong to the mesh by publishing a ServiceEntry and VirtualService (in infra-lb) for the LoadBlancerIP. Yes, not for the mariadb Server, but for the service.
piVersion: networking.istio.io/v1beta1 kind: ServiceEntry metadata: name: mariadb-metalllb-lab-istio spec: hosts: - mariadb-metalllb-lab-istio.interne.lab.montreal.ca # not used addresses: - 10.224.3.102 # LoadBlancer VIPs obtained through MetallLB ports: - number: 3306 #name: tcp-maria name: maria protocol: TCP location: MESH_EXTERNAL resolution: NONE
apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: routetcp-maria-mettallb spec: hosts: - mariadb-metalllb-lab-istio.interne.lab.montreal.ca tcp: - match: - port: 3306 route: - destination: host: haproxy-mariadb-srv port: number: 3306
This will correct connexion 3. That is, the centOS pod that belongs to the mesh will now be able to connect to mariadb using the LoadBalancerIP. But, external connexion still do not work, connexion one for instance.
I have being able to make it work by adding a PeerAuthentication object that deactivates mTLS altogether :
apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default spec: mtls: mode: DISABLE # portLevelMtls: # 3306: # mode: DISABLE
Notice that I cannot have
DISABLE. It’s just ignored for some reason.
Can anyone shed light on this behaviour ? Why are there differences for cluster-ip and loadbalancer ip when coming in from the mesh. And differences between mesh clients and outside-of-kubernetes clients on the loadbalancer-ip ? Is it possible to have my cake and eat it too ? That is have mTLS for mesh clients and no TLS for external clients ?