Configure which resources a gateway watches

Hi! I see that for sidecars in an Istio service mesh, it is possible to configure which config is pushed to the istio-proxy using the Sidecar resource. This is something we use as our mesh has many 1000s of pods. But the Sidecar resource is not applicable for gateways, as documented in Istio / Sidecar
Is it possible to limit which configuration is pushed to each gateway?
The reason is that in one cluster, we have multiple gateways, one in each namespace, and each gateway only fronts the services in the same namespace. So pushing config from other namespaces to the gateway is just overhead, and a lot of it. So any way to limit the config that is pushed to the gateways would greatly improve the performance and stability.