I’m currently struggling a bit and i think i maybe misunderstand how some parts of istio work. i’d appreciate it if someone could help me out or point me in the right direction thanks! Apologies for the lengthy post.
Current Setup:
I’m currently running an aks cluster with istio (1.1.8, mtls enabled). everything runs fine, communication between services and so forth. now i’ve created a different namespace which should run services outside of istios service mesh (in my case, as a start, i’ve deployed rabbitmq). the services start and run fine from the outside (e.g. i can connect to rabbitmq, publish and subscribe, from my local machine and other clients).
The problem:
Now i’d like to also use those services from a pod within the service mesh (with the sidecar injected). I thought that all i’d need todo is to define a destinationrule with tls: disabled for the host dns (..svc.cluster.local). but the service with the sidecar injected can’t connect.
i’ve tried adding a serviceentry, changing the mtls mode to permissive, using the ip directly, no success.
if i run the same pod, without a sidecar (everything else untouched) it connects and works as expected.
as a last step i tried to run the rabbit service from within the mesh (original namespace, sidecar injected, …). this lead to a different issue, the service with k8 peer discovery does not even start because it can’t reach the kubernetes api server (kubernetes.default.svc.cluster.local). i’ve checked with istioctl authn tls-check and it says it would use the default/default auth policy (which just contains the mtls mode strict (or permissive tried both) as well as the destination rule api-server/istio-system (which has the tls mode disable).
So it seams to me, that as soon as i inject the sidecar, that i cant get any traffic to anything that is not within the service mesh. That brought me to my istio setup, which configures the proxies to only include a certain ip range (helm values: global -> proxy -> includeIPRanges). Which gave me the idea that maybe i could exclude the service port (rabbit) of one of my services that tries to connect. i’ve tried adding the annotation traffic.sidecar.istio.io/excludeOutboundPorts to my client service, but unfortunately no luck either.
It’s hard to follow without any example configuration, but I have to ask: why are you using includeIPRanges? It sounds like you set this before you hit the problem; but this setting is specifically there to circumvent Istio controlling all egress traffic. And there are other ways, too: simply adding a ServiceEntry will suffice, usually. Or, if you don’t want Istio to manage any outgoing traffic, you can set the outboundTrafficPolicy.mode to ALLOW_ANY: https://istio.io/docs/tasks/traffic-management/egress/egress-control/#envoy-passthrough-to-external-services
i’d be glad to clear things up with my configuration, i’ve just tried so many different things and different approaches that i’m not sure what would help without making it even more difficult to follow. anything in particular that would help?
regarding your question about the includeIPRanges. when i configured the cluster (with istio 1.0.2) another developer added a service that had to interact with some external databases (one azure sql and one mongodb atlas running on aws). for whatever reason, the connection to the mongo cluster did not work out with service entries and so forth and because the whole thing was on a tight schedule, i restricted the ip ranges to let the service connect directly. by now the outboundTrafficPolicy mode should be per default ALLOW_ANY if i’m not mistaken, so maybe i should revisit this at some point too.
i do have an update on my initial issue though. i got one approach to work, though i don’t think the steps taken are considered “best practice”. as described above, on approach i’ve tried is to run rabbitmq as part of the mesh (with a sidecar), which did not work because it could not connect to the api server (theres the istio default destination rule in place here). so, what i did is, i’ve added the api server to the excludeIPRanges. which allowed rabbit to connect and the communication mesh internally works as expected. i’m just not sure why the connection would fail in the first place through the sidecar.
thats the destination rule in use (checked with tls-check):
thanks! Apologies for the lengthy post.