An update to the case - after disabling Cloudflare (going from orange to grey cloud - direct IP connection) and having the client connect directly to ingressgateway the issue has yet to re-surface.
It seems like the issue occurs when Cloudflare speaks HTTP/2 to the origin server, something that according to their own documentation should never happen.
I tried putting a question in to the Cloudflare community page regarding this too, but no answers so far.
Now all traffic between the browser and istio-ingressgateway is http/2 and the only delays I have seen so far is when the backend services are taking their time to reply.