perhaps i’m missing the forest for the trees, it would be great if you could help me out with an alternative or verify that i understood things correctly:
I want to establish a connection to a resource outside of the mesh, say, an oracle database on port 1521.
So i created a serviceEntry like this, and it works fine.
The result then is chaos, as Istio/Envoy uses only the port to identify the routing for TCP traffic if no addresses are explicitly set, according to the describtion of the addresses parameter in (https://istio.io/latest/docs/reference/config/networking/service-entry/#ServiceEntry).
It even routes traffic to the wrong database based on the sidecar access logs, even though the application is configured on the correct hostname. Ugh!
Now, according to the documentation i need to identify unique CIDRs or IPs for each of those DBs and list these in the addresses parameter in order to avoid this issue.
That seems to work.
But what if one can’t do this? What if they’re different instances within the same CIDR block, or those might change? This seems to defeat the purpose of using DNS, at least partially. That seems like a very severe limitation to me.
How would you solve this, are there mechanisms within istio/envoy themselves to work around this issue that do not require changes to the infrastructure outside of it?
I just can comment on what you just wrote without knowing the Istio-internal implementation, but…
as your Service is a TCP service, the host name to address translation is already happening within your service. Therefore the outgoing traffic only carries the IP and port information and the rest of the stream is fully opaque to Envoy. As there is host mapping in Istio, the IP might not even be the one of the external service. For example when you use the egress gateway.
Now… is there a way around. I think so. Your service is an OracleDB which supports TLS . And TLS has the SNI (Server Name Indicator; I think starting with 12.2 - Oracle support should be able to help). So, if you change your ServiceEntries to be for protocol TLS instead of TCP, you should be able to get it working as then Envoy has something to work with again in the otherwise opaque stream. Otherwise we’d see the very same problem with HTTPS services as well (and that would be a known limitation)
Thanks! I can confirm that setting the protocol to TLS ensured that the correct IP appears in the sidecar logs.
There were some other issues with the DB connection then in my case, but that seems unrelated to Istio.
I found another way to resolve this issue though:
It appeared in my case only when the serviceentry was set to use resolution=DNS.
When using resolution=NONE, i.e. letting the DNS name be resolved by the client in the application instead of doing it again in the sidecar, then apparently the sidecar behaves quite different, in a way that doesn’t seem to be vulnerable to this problem. At least i couldn’t reproduce it any more since then, and given the description thereof it also makes sense that it wouldn’t happen anymore.
Assume that incoming connections have already been resolved (to a specific destination IP address). Such connections are typically routed via the proxy using mechanisms such as IP table REDIRECT/ eBPF. After performing any routing related transformations, the proxy will forward the connection to the IP address to which the connection was bound.
Caveat:
According to the description of “resolution” in ServiceEntry, it’s still advisable to add addresses to the service entry then:
Service discovery mode for the hosts. Care must be taken when setting the resolution mode to NONE for a TCP port without accompanying IP addresses. In such cases, traffic to any IP on said port will be allowed (i.e. 0.0.0.0:<port> ).