Connection stability issue with east-west gateway in Azure


I have two AKS instances, each sits in a subnet. Both subnets are in the same vnet. After the instances were provisioned, I followed this blog and set up east-west gateway connections between the two clusters.

Then I followed the next steps to verify the set-up.

After I executed the command:

kubectl exec --context="${CTX_CLUSTER1}" -n sample -c sleep \
    "$(kubectl get pod --context="${CTX_CLUSTER1}" -n sample -l \
    app=sleep -o jsonpath='{.items[0]}')" \
    -- curl -sS helloworld.sample:5000/hello

I can see the version number alternate between 1 and 2. I think this indicates that my east-west gateway is set up correctly.

But after a while, when I retried the same command, I observed the following issues:

  • the command staled
  • connection reset

I checked all the ingressgateway and istiod pods but could not find any obvious error. If I keep retrying, these issues will resolve by themselves.

It seems that the connections between the two clusters are not very stable, and it is most likely to observe these issues if you let the system stay idle for a while.

How can I check which component is causing these issues?