AWS NLB with service externalTrafficPolicy = Local

Networking
1

Hi Team,

I’d like to use AWS NLB for our ingressGateway. And the only way to preserve client PI is to set Service externalTrafficPolicy=Local. And as it describes in K8S Service Doc

Nodes without any Pods for a particular LoadBalancer Service will fail the NLB Target Group’s health check on the auto-assigned .spec.healthCheckNodePort and not receive any traffic.

And this works as expected.
The default Health check params for Target Groups in Kubernetes 1.15.6:

  • Unhealthy threshold is set to 3
  • Interval 30s.
  • Timeout 6s

And seems it brings an issue in case we have enabled HPA for ingressGateway. When hpa scales down the pod - there is a delay in NLB/TargetGroups when it moves instance from healthy → unhealthy. And for 90+ seconds NLB routes traffic to this nodes.

What would be a proper solution or workaround here?

2

Seems Health Check params were reduced - PR

3

I have the same scenario, is there any solution or workaround?
looks like the PILOT_SIDECAR_USE_REMOTE_ADDRESS does not solve the issue too.