mTLS configuration ignored

Security
1

I recently started playing with Istio and installed it on a GKE cluster. I have an Ingress that is connected to an HTTP LB on GCP and uses managed certificates to enable SSL. I’ve created 4 small pods, where a frontend talks to 3 backend pods and to test traffic management I deployed 2 separate versions of the app:

This all works fine and when you visit the URLs you can see the different versions of the “app”. However, I looked into enforcing mTLS and for some reason this doesn’t seem to work at all. The URLs I configured in the frontend pod are all HTTP based.

Initially, I enforced mTLS across the board:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: STRICT

But that didn’t break any of the links between pods. I then applied it to the namespace, called books:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: books                      
spec:
  mtls:
    mode: STRICT

Again, same result, the application remained up and running.

I updated the DestinationRules to add the mTLS configuration:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: reviews-dest-rule
  namespace: books
spec:
  host: books-reviews
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL

To test that the mesh was actually being triggered and I wasn’t using plain old Kubernetes, I added an AuthorizationPolicy to the backend pods and blocked all communication (empty rules-block). When I refreshed the application, I received errors, so I interpreted that as a sign that my configuration was taken into account.

I checked one of the sidecars and when I look at the logs, I can see the following entry:

2021-01-27T15:30:07.615537Z	warning	envoy config	adding listener '0.0.0.0:15006': filter chain match rules require TLS Inspector listener filter, but it isn't configured, trying to inject it (this might fail if Envoy is compiled without it)
2021-01-27T15:31:17.815841Z	warning	envoy filter	mTLS PERMISSIVE mode is used, connection can be either plaintext or TLS, and client cert can be omitted. Please consider to upgrade to mTLS STRICT mode for more secure configuration that only allows TLS connection with client cert. See https://istio.io/docs/tasks/security/mtls-migration/
2021-01-27T15:31:17.816965Z	warning	envoy filter	mTLS PERMISSIVE mode is used, connection can be either plaintext or TLS, and client cert can be omitted. Please consider to upgrade to mTLS STRICT mode for more secure configuration that only allows TLS connection with client cert. See https://istio.io/docs/tasks/security/mtls-migration/

Either I completely misunderstood something (very likely), but it’s my assumption that once you enable STRICT-mode, it shouldn’t allow any HTTP traffic anymore and enforce HTTPS instead. However, is there some funky translation happening under the hood that I can’t figure out? I can’t find anything in the docs that I can look at or that I’m missing, as these are the only resources that I can find in the articles.

Or have I completely misunderstood how this should work? :slight_smile:

2

I’m the muppet here I think. I did some more investigation yesterday and I think I misinterpreted the training videos I was following.

For Pod to Pod communication inside the mesh, it’s the Envoy proxy that actually takes care of augmenting requests with mTLS and ensuring that this happens from both ends. So this is completely transparent for the Pod, but that traffic interruption is happening at sidecar-level. I want to look into this in more detail by looking at the tcpdump, but haven’t had the time yet to do that.

Where it does kick in, is for requests that are coming in from external (i.e. outside of the mesh). I ran some tests yesterday with pods running in different namespaces (and didn’t have the Istio injection applied) and those requests were blocked.

3

To further clarify: The Istio-proxy that includes envoy IS a part of the pod and is deployed as a sidecar container in the pod. As part of pod bringup, the istio-proxy fetches its own certificates that are used for pod to pod communication. These certificates are also used for all TLS based communication from your ingress gateway to the backend pods. Also, note that with the later versions of Istio, mTLS is enabled by DEFAULT (even in the absence of the destination rule), so it should be transparent to you.