Are there any performance tuning guidelines for terminating TLS with Istio ingress?
A bit of background:
Out of the box, we’re seeing that istio-ingressgateway pods run extremely hot when terminating TLS. Under load, the ingress gateways are creating a major bottleneck for https traffic, and we haven’t had any luck tuning them to relieve the problem. For example, with a load test that ramps up to 100 concurrent users pinging a noop healthcheck route, we see latency ramp up from a start of 60ms to 500ms+.
We’ve tried both bumping up the CPU resource requests (e.g. 1000m) and the maximum number of ingress gateway pods in the HPA (e.g. 150 max), but haven’t found any configuration that makes a meaningful difference.
(As a workaround, we’re currently looking at moving TLS termination to our ELBs. We’re on EKS using Istio 1.0.2. We’ve tried Istio 1.1 as well, but that didn’t help.)