We have Kubernetes HPA enabled for our pods as such
spec: maxReplicas: 4 minReplicas: 1 targetCPUUtilizationPercentage: 80
and pods resource requests configured as such:
app: resources: requests: cpu: 75m
Using this configuration we would expect the HPA to scale a new pod at 60m ( 80% of 75m). However, we found that with the Istio sidecar injected scaling does not happen when expected at 60m.
I can see in the pod both containers have resource requests as such:
app: resources: requests: cpu: 75m istio-proxy resources: requests: istio-proxy: 100m
I understand that k8s calculates the HPA scale out differently when you have multiple containers in a pod.
At what point will HPA scale a new pod with Istio sidecar injected, and what should the
targetCPUUtilizationPercentage be adjusted to to reach the same effect as prior to sidecar injection