In-mesh communication entry point

Hi there, I am trying to find simple but complete examples of 2 or more microservices in the same namespace using the mesh to communicate to each other.

I am struggling to understand the entrypoint for communication here - I’m trying to set up a canary deployment, and everything works as long as I use the gateway, but I shouldn’t be using the gateway for in-mesh communication, right? The canary routing rules are at the virtualservice level, but from multiple posts, it seems to me that virtualservices don’t get exposed in the internal DNS normally, so I can’t use service discovery. A lot of Istio documention mentions service discovery, but there’s no page indicating how this is configured or how it works. I tried to add “mesh” as a gateway, but that was accepted yet didn’t do anything.

The way I’m trying to test this (which could also be wrong) is to kubectl exec ... sh into one of my app containers and curl/wget a url for the virtualservice (name, host, route name, etc, none worked).

Am I doing something wrong?