TL;DR
I created a multi primary cluster setup and try to reach an identical service deployed in both clusters through a custom host name e.g. from standard my-app.default
to custom my-app.service.consul
. But I get only a response from the local cluster and not both clusters.
Setup
I created two clusters and configured a multi primary cluster setup. Each cluster has an identical service that are both reachable from any cluster. E.g. calling the service from cluster A will return responses from both clusters:
cluster-a$ curl my-app:8080
"hello from cluster a"
"hello from cluster b"
...
I wanted to extend my setup by mapping the service DNS to a custom one by using a Service Entry and a modification in the istio-operator:
ServiceEntry definition:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: my-app
spec:
hosts:
- my-app.service.consul
location: MESH_INTERNAL
ports:
- number: 8080
name: http
protocol: HTTP
resolution: DNS
endpoints:
- address: my-app.default
ports:
http: 8080
- address: <my-aws-lb>.elb.amazonaws.com
ports:
http: 15443
subjectAltNames:
- "spiffe://cluster.local/ns/default/sa/my-app"
Istio operator modification:
...
meshConfig:
defaultConfig:
proxyMetadata:
# Enable basic DNS proxying
ISTIO_META_DNS_CAPTURE: "true"
ISTIO_META_DNS_AUTO_ALLOCATE: "true"
...
This allows me to call the service by the host name specified in the service entry. But I only get a response from the local cluster, but not both clusters:
cluster-a$ curl my-app.service.consul:8080
"hello from cluster a"
"hello from cluster a"
...
From east west gateway logs I can see this message:
[2021-10-22T12:49:16.399Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 100.96.43.64:15443 100.96.36.28:50136 - -
While the istio proxy has the following destinations registered:
100.71.120.168:8080 HEALTHY OK outbound|8080||my-app.service.consul
100.96.43.218:8080 HEALTHY OK outbound|8080||my-app.default.svc.cluster.local
<AWS_LB>:15443 HEALTHY OK outbound|8080||my-app.default.svc.cluster.local
<AWS_LB>:15443 HEALTHY OK outbound|8080||my-app.service.consul
At this point I am not sure if there is a specific configuration missing, or if it just doesn’t work by design. Any suggestions?
Using Istio Version: 1.11.4