Communication between gateway in strict mTLS setup


I have istio configured with strict mTLS setup globally.
When I created kfserving’s inferenceservice, I noticed that it creates additional virtualservice that defines traffic routing from istio-ingressgateway to cluster-local-gateway

The call from external gateway (istio-ingressgateway) to internal gateway (cluster-local-gateway) is failing when strict mTLS is enabled with some SSL error message (SSL routines:ssl3_get_record:wrong version number).
it’s fine when I change to permissive mTLS.

I tried few configurations such as

  • adding DestinationRule for cluster-local-gateway port: 80 to disable TLS, or
  • set port 80, in Gateway CR to use ISTIO_MUTUAL
    none is working.

Can anyone help me understand how communication between gateway works?
and how can I make strict mTLS works with this routing configuration?