How to debug ingress gateway for hanging requests

Hi :wink:

I have an issue with requests freezing/hanging on ingress gateway using TLS (mode simple).
Happens for requests with payloads like 700KB but not for bigger ones strangely.
I have mTLS enabled and my workload has sidecar enabled. I’m not able to reproduce this problem over HTTP or with internal (pod A to pod B) communication - both pods have sidecars.
I’m looking for some help/tips how to debug this topic :wink:

I have gathered more detailed information with steps to reproduce to get the better context in the issue https://github.com/istio/istio/issues/23855

The httpbin is used for simple debugging/reproducing but I have the same results with my applications.

What I did so far :
I have checked logs from ingress proxy, istio-proxy of my app, I have verified mTLS setup.
I also took a look to kiali and dashboards, requests form ingress as well as httpbin pod are observed like below but when I checked the request graph for entire request flow, from time to time I see that the graph adds new entry called "httpbin.pm.svc.cluster.local’ and request goes to that node. It correlates in time with requests that are “freezing”.


I’m currently stuck so I will be grateful for any tips :wink: