When trying to change outboudTrafficPolicy to REGISTRY_ONLY we ran into an issue where ither pilot or the istio-proxy sidecars would reject updates with an error similar to:
Error adding/updating listener(s) 0.0.0.0_443: multiple filter chains with overlapping matching rules are defined
From that point any ServiceEntry created would not be in the istio-proxy registry (checked with istioctl proxy-config cluster) and the sidecar would not let the connection go through, as expected.
My current hypothesis is that multiple ServiceEntry configurations for the same hostname might be causing the overlap because different teams deploy to different namespaces, but I can’t really reproduce/prove it. We are also enforcing setting exportTo: '.' in the ServiceEntry to keep it scoped only to the namespace where it is deployed.
We had similar issues before when we were setting TLS resolution and people would deploy bad certs or reference certs that didn’t exist (typo or something), this caused failures in the pilot push process as well as in ingress (sometimes no ingress traffic was allowed, other times only HTTP worked).
Anyways, I’m hoping to see if someone else ran into this error before or if anyone has ideas on what to look for in the configuration
For context: we are running on istio 1.2 and kubernetes 1.13 (EKS).
Ok this is what I found, maybe it can help someone (or my future self )
We first made sure that all ServiceEntry configuration deployed had exportTo: set to - "." to scope it only the namespace where it runs. That reduced the amount of messages we saw in the log aggregator by almost 5x.
After that I noticed only two kind of error messages were showing up, the first one:
Error adding/updating listener(s) 0.0.0.0_443: error adding listener ‘0.0.0.0:443’: multiple filter chains with overlapping matching rules are defined” >
And the second one (100x less common the the first):
Error adding/updating listener(s) 0.0.0.0_443: error adding listener ‘0.0.0.0:443’: multiple filter chains with the same matching rules are defined
Looking at the Listener Manager code the two error messages I was seeing did not seem like they would cause problems. The first one has a nice little comment with the reasoning of what happened in that particular code branch and it talks about the fact that there already is a filter configured. The second message is inside an if that checks if the filter is not at the end of the chain and is inside a loop that goes through all configured filter chains to find if it needs a TLS Inspector.
None of these two seemed to indicate a cause to the problem I was seeing before, where it would prevent a ServiceEntry from being created. I could also confirm that all hosts where being added by checking the registry of the pod sidecars using istioctl proxy-config cluster.
I’m still not sure what trigger those warnings as I could not reproduce the problem in other clusters but they seemed to be just warnings.
)