Context: We used to use Istio’s rate limit capabilities until they got deprecated in Istio 1.5. My understanding is, that the current recommendation is to use Envoy Rate Limits (there is also some new documentation explaining how). At least until rate limit capability will be brought back to Istio (tracked with 20952).
I’m trying to understand if the current recommendation of using Envoy rate limits supports the following.
Scenario we would like to support: We would like to mitigate the following issue by having conditional rate limits based on source service name/id:
Let’s say we have service A and service B. Both call service C:
A => C
B => C
We have a bug in service A, resulting in A sending lots of requests to C. C gets overloaded by it, impacting also B that is now unable to retrieve a response from C:
A == send lots of requests to ==> C
B => C …B gets error responses from C as C is overloaded with requests.
We would like to apply a conditional rate limit for service C: “max 10 requests per second, for a given source service”:
A == sends lots of requests to ==> C …however, Envoy returns error codes as soon as the 10req/sec is hit, preventing C from getting overloaded with requests.
B => C …these requests succeed, as C is not overloaded, thanks to Envoy throttling requests from A.
I’m trying to connect the dots and understand if the above scenario is currently possible and if yes, how it can be accomplished.
If anyone has any pointers on how to implement the above with Istio/Envoy, that would be great.