I could use some help with ideas on how to debug or understand how the sidecar manages connections with the app…
I’m able to trigger connection limits in the app (won’t accept more than 1024 open connections at a time) which basically means the sidecar is not closing connections when requests are complete (I know the connections are closed by the client). It’s easy to trigger this using some load testing tools like cassowary to open a bunch of requests. The app gets into a state where healthchecks start failing and k8s restarts the pod. By all accounts the requests are short lived and the sidecar should be closing them when the upstream closes.
Is there something that can be tuned around this? Never had the problem with this app when it was nginx opening the connections to the app.
The connections are http and I’m using istio 1.12.2. The topology is AWS NLB → istio ingress gateway → sidecar → app.