Istio upgrade leads to duplicate listener error

We’re having issues upgrading from istio 1.4.3 to 1.6.5.

It seems like one gateway is being configured with the wrong port.

Logs/config can be found here: GitHub - nais/istio-issue

The gateway: ingress-gateway-dev-adeo-no for some reason creates a listener on port 445 and 82,
which is actually configured for the gateway: ingress-gateway-dev-nav-no. It should have been configured with 443/80 (this can be seen here)

This gives the following error in istiod:

2020-08-04T11:31:23.888558Z     warn    ads     ADS:LDS: ACK ERROR router~10.6.82.38~istio-ingressgateway-6d559ffbf4-9xpw9.istio-system~istio-system.svc.cluster.local-2 Internal:Error adding/updating listener(s) 0.0.0.0_82: duplicate listener 0.0.0.0_82 found
0.0.0.0_445: duplicate listener 0.0.0.0_445 found

Any advice on troubleshooting this issue would be greatly appreciated.

1 Like

We found out the issue was due to multiple services listening on the same ports (80, 443) which made buildGatewayListeners mix them up. Getting rid of the warning below also resolved our issue.

2020-08-04T11:31:23.263867Z warn buildGatewayListeners: found 6 services on port 80: map[istio-ingressgateway-adeo.istio-system.svc.cluster.local:{} istio-ingressgateway-devadeo.istio-system.svc.cluster.local:{} istio-ingressgateway-devnav.istio-system.svc.cluster.local:{} istio-ingressgateway-nais.istio-system.svc.cluster.local:{} istio-ingressgateway-nav.istio-system.svc.cluster.local:{} istio-ingressgateway.istio-system.svc.cluster.local:{}]

2020-08-04T11:31:23.264223Z warn buildGatewayListeners: found 6 services on port 443: map[istio-ingressgateway-adeo.istio-system.svc.cluster.local:{} istio-ingressgateway-devadeo.istio-system.svc.cluster.local:{} istio-ingressgateway-devnav.istio-system.svc.cluster.local:{} istio-ingressgateway-nais.istio-system.svc.cluster.local:{} istio-ingressgateway-nav.istio-system.svc.cluster.local:{} istio-ingressgateway.istio-system.svc.cluster.local:{}]

We were confused initially because this didn’t happen in any other clusters, most likely the order in which the gateways gets processes matters here. We were however not able to reproduce this issue even with identical configuration in another cluster.

1 Like