404 when using a FQDN with final dot

1

After setting up sidecar proxies on a few Kubernetes deployments I found out that when trying to do http requests to internal services using their FQDN ending in a dot (eg. foo.default.svc.cluster.local.), I get an immediate 404 response from envoy. I confirmed this with a tcpdump: when using the final dot FQDN, envoy sidecar doesn’t do the actual request to the other service, instead, immediately returns 404 to the caller.

An interesting test is to use curl and wget in this case – while curl removes the final dot from the Host header, wget does not, hence, if I do curl http://foo.default.svc.cluster.local./ I get a 200, whereas a wget http://foo.default.svc.cluster.local./ fails with 404.

Digging deeper, I noticed that istio-proxy is started with --domain $(POD_NAMESPACE).svc.cluster.local (no dot), which looks good. I’m not sure changing this is something I want to do, so, I am wondering if there’s a way to support the last dot in a FQDN.

João

2

Yes as you noted, the Host header cannot have the trailing dot

3

How do I resolve this issue? I have an existing deployment where I want to enable istio and due to this issue, inter service communication is breaking.

4

Just curious, why do you want an address with the trailing dot?

5

With a trailing dot, the dns lookup is faster and looks it up only for the specific host in that domain. Otherwise it ends up looking up in all the domains. We standardized this in our deployment in order to reduce load on kube-dns.

6

The table on https://github.com/istio/istio/issues/13710 clearly explains the dns lookup count without the trailing dot.