Hi,
I have a istio service mesh installed on kubernetes cluster with mlts enabed (values.global.mtls.enabled=true values.global.controlPlaneSecurityEnabled=true).
I am using ambassador as the ingress and followed this guide to integrate with istio.
Now, everything seems to work fine but sometime I get 504/503 error when accessing my services. I checked the access logs for istio-proxy (sidecar) and it seems that it can’t route the request to my service container.
I get the following access logs in case of a successful request:
[2020-01-31T09:23:37.560Z] "- - -" 0 - "-" "-" 2386 216 23 - "-" "-" "-" "-" "127.0.0.1:8080" inbound|8080||documents-service.mesh.svc.cluster.local 127.0.0.1:36944 10.36.11.19:8080 10.36.8.15:58808 - -
which seems to indicate that it route the request to the localhost:8080 on my pod which where the service is running.
But for the 504/503 response, I get the following logs:
[2020-01-31T09:18:25.823Z] "- - -" 0 - "-" "-" 1171 0 2999 - "-" "-" "-" "-" "10.36.8.19:8080" InboundPassthroughClusterIpv4 127.0.0.6:39585 10.36.8.19:8080 10.36.8.10:39630 - -
In this case, it seems it can’t redirect the request to localhost:8080.
Note: I have two pods in my deployment. 10.36.8.19 and 10.36.11.19 are IPs of the pods.
Anyone has an idea what might be happening here?
Thanks.