DestinationWeight on route doesn’t have a valid service (host not found) - Kiali error pointing at Istio

Policies and Telemetry Kiali
1

Not sure if this is the right place but I asked on CNCF’s Istio Slack channel and got no responses so maybe it would make more sense to put it here.

I am trying to debug an access error I can see in Kiali and I am sort of stuck.

The error is “DestinationWeight on route doesn’t have a valid service (host not found)”
and the spec part of VirtualService it points at looks like this:

spec:
  hosts:
    - argocd.private.dev
  gateways:
    - argocd-private-gateway
  tls:
    - match:
        - sniHosts:
            - argocd.private.dev
          port: 443
      route:
        - destination:
            host: argocd-server.argocd.svc.cluster.local
            port:
              number: 443

As described here Validation | Kiali this should mean that fqdn “argocd-server.argocd.svc.cluster.local” is not reachable or does not exist, but I can ping it from a pod created in a different namespace than the one it is in and it resolves fine - additionally if I make the service as nodeport I can get to it fine (so it works fine) and I am not sure why Istio cannot resolve/get to it?

Istio 1.17 installed via helm chart.

Has anyone encountered this problem? Any hints where I can look for clues?

To add a bit more of context, I’ve tried with kubectl sniff and can see that traffic gets to the istio-ingress but nothing gets past it to the service/pod and it returns

HTTP/1.1 503 Service Unavailable
server: istio-envoy
2

I just found the reason, thanks to this answer on StackOverflow

After listing mesh services the problem was obvious and was related to the fact that I’ve changed trust domain of the cluster and this caused all service entries to be prepended with it instead of the standard “cluster.local” part so after I’ve changed this in my VS definition it started working right away.

I guess maybe this type of problems with trust domain should be documented somewhere? or maybe it is I just have not came across it …

1 Like
3

Maybe this would have helped… Validation | Kiali