How does kubectl port-forward bypass envoy mTLS enforcement?


I have an architectural question for which I haven’t been able to find an answer in the Googles.

If I’ve enabled mandatory mTLS, and I deploy an app to the mesh, then I cannot connect to it via a Kubernetes LoadBalancer service. This is expected, as I’m not doing mTLS in my connection.

However, if I then do a kubectl port-forward, then I can connect to the app through the locally forwarded port. I’m not sure how that works since the port-forward is sending my connection to the exposed port on the Pod in the same way that the service was. Does Istio/Envoy explicitly not implement policy against ports forwarded this way? If so, is there documentation explaining that (or how it works)?


I’m not 100% on the details of how port-forward works, but it definitely is not “the same way that the service was.”

Port-forward involves the Kubernetes API server opening a SPDY connection to Kubelet, and Kubelet opening the stream to your pod. My suspicion is that the kubelet to pod stream is done in such a way as it bypasses the iptables rules that normally redirect IP packets thru the proxy. It also depends on the container runtime (e.g. Docker, CRI-O, rkt).

I’ll ask around the office and see if I can get a more detailed answer from Kube experts here.

1 Like

Much appreciated, Spike. I expect that understanding how this works will help me understand Istio/K8s networking in general.


Port forwarding works by essentially running a ‘docker exec’ into the container and running socat against the port on localhost. Since this is over localhost it is not captured by envoy.



The code you are looking for is below, it is entering the containers network namespace (nsenter) and within that running socat against TCP4:localhost:%d


1 Like

Perfect! Thanks @kconner!

Would you accept a PR explaining this in the Istio documentation? If so, which page would you suggest editing?


I don’t know of a good existing place to include this information. What kind of thing are you wanting to say? Maybe we can find a good home for it.

cc @geeknoid


For this kind of content, I’d shoot for either a FAQ page (/help/faq), or something in the operations guide (/help/ops)


I have a problem is quite different.
When I execute the command
kubectl port-forward -n monitoring prometheus-prometheus-operator-prometheus-0 9090 on my cluster Kubernetes (AKS) it gives me this result
Forwarding from -> 9090
Forwarding from [:: 1]: 9090 -> 9090
and when I put the URL on my browser, I have no result.

$ kubectl get pods -n monitoring
alertmanager-prometheus-operator-alertmanager-0 2/2 Running 0 11h
prometheus-operator-grafana-8549568bc7-zntgz 2/2 Running 0 11h
prometheus-operator-kube-state-metrics-5fc876d8f6-v5zh7 1/1 Running 0 11h
prometheus-operator-operator-75d88ccc6-t4lt5 1/1 Running 0 11h
prometheus-operator-prometheus-node-export-zqk7t 1/1 Running 0 11h
prometheus-prometheus-operator-prometheus-0 3/3 Running 1 11h

What should be done ? or how to forwarding prometheus from my public cloud to my local machine.
Thank you