There was already a similar question here Istio Load balancing by custom metrics but since the answer wasn’t really helpful, I’ll post it again with a bit more details.
Currently, Istio uses Envoy for its data plane (network proxy handling routing and load-balancing). Istio “exposes” three load balancing algorithms available with Envoy, which are
- random
- weighted, and
- least request
However, there are a few more that Envoy supports which does not seem to be exposed via Istio,
In a highly distributed system with a possibly heterogeneous topology, we would like to have the ability of a more sophisticated global load balancing scheme, ideally based on performance and load metrics provided by upstream services.This is especially important for us when deploying to public clouds since
- a service instance running in a pod could be negatively impacted by “noisy neighbors” and we would like to route less traffic to such underperforming pods, and
- data centers of public clouds have typically several generations of CPUs available with different performance characteristics. It is possible that some pods could be scheduled to run on “faster” CPUs than others and we would like to route traffic according to performance characteristics of such pods.
One idea to achieve this would be to implement a service that taps into performance metrics reported by service pods and based on that, determines weights for load balancing which then gets fed back to Envoy via xDS for “weighted load balancing”.
Question 1: has anybody already thought about implementing something like that and if yes, would that be sharable with the community (GitHub link)?
Question 2: if the answer to question #1 was “no”, would there be interest to propose such a feature and implement it as part of Istio?