Hello all,
We activated istio in our lab in PERMISSIVE mode and a few services broke. One of them is a loadbalancer for mariadb servers that live outside of kubernetes. The loadbalancer live in pods and the service fronting them is using a LoadbalancerIP obtained through MetalLB.
In schema that follow, the Mariadb servers live in vms. Before injection with with the side car, everything worked just fine. Connexion path 1,2, 3 and 4 worked juste fine.
Once side-car injection has happened, a few connexions stoped working.
That is, connexion 1 and 3 that try to talk to the LoadBalancerIP (which is obviously not a clusterIP) do not work. It obviously has to do with the server side protocol that is mysql. I have modified the service in the infra-lb namespace to tag correctly the service with either or both appProtocol field or to make sure the name respects name: <protocol>[-<suffix>] syntax.
To no avail. The behaviour is still different between the two IPs. By the ways, CentOS-tools is simply a centos image with lots of tools that we have in each of our kubernetes clusters so we can comfortably tests things.
Notice that the CentOS-tools pod can talk directly to mariadb through connexion without any issues.
I can fix the problem for the CentOS tools image which belong to the mesh by publishing a ServiceEntry and VirtualService (in infra-lb) for the LoadBlancerIP. Yes, not for the mariadb Server, but for the service.
piVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: mariadb-metalllb-lab-istio
spec:
hosts:
- mariadb-metalllb-lab-istio.interne.lab.montreal.ca # not used
addresses:
- 10.224.3.102 # LoadBlancer VIPs obtained through MetallLB
ports:
- number: 3306
#name: tcp-maria
name: maria
protocol: TCP
location: MESH_EXTERNAL
resolution: NONE
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: routetcp-maria-mettallb
spec:
hosts:
- mariadb-metalllb-lab-istio.interne.lab.montreal.ca
tcp:
- match:
- port: 3306
route:
- destination:
host: haproxy-mariadb-srv
port:
number: 3306
This will correct connexion 3. That is, the centOS pod that belongs to the mesh will now be able to connect to mariadb using the LoadBalancerIP. But, external connexion still do not work, connexion one for instance.
I have being able to make it work by adding a PeerAuthentication object that deactivates mTLS altogether :
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
spec:
mtls:
mode: DISABLE
# portLevelMtls:
# 3306:
# mode: DISABLE
Notice that I cannot have mtls to PERMISSIVE and portLevelMtls to DISABLE. It’s just ignored for some reason.
Can anyone shed light on this behaviour ? Why are there differences for cluster-ip and loadbalancer ip when coming in from the mesh. And differences between mesh clients and outside-of-kubernetes clients on the loadbalancer-ip ? Is it possible to have my cake and eat it too ? That is have mTLS for mesh clients and no TLS for external clients ?
