Istio in Azure AKS - Outbound traffic issues over 15001 port while connecting to Azure Redis Cache

Networking
1

Hi Team,

We are facing issues on 15001 port in istio deployed in Azure AKS.

Currently we have deployed istio in AKS and trying to connect to Azure cache redis instance in cluster mode. Our Azure redis instance is having more than two shards with SSL enabled and one of the master node is assigned on port 15001. We were able to connect to Azure redis from AKS pods over ports 6380, 15000, 15002, 15003, 15004 and 15005 ports. However when we try to connect to over 15001 we see some issues. When we try to connect to redis over 15001 port from a namespace without istio sidecar injection the connection is working fine.

Below are the logs from rediscli pod deployed in our AKS cluster.

Success case:

redis-cli -h our-redis-host.redis.cache.windows.net -p 6380 -a our-account-key --cacert "BaltimoreCyberTrustRoot.pem" --tls ping

OUTPUT:
Warning: Using a password with ā€˜-aā€™ or ā€˜-uā€™ option on the command line interface may not be safe.
PONG

We are able to connect over all ports - 6380, 15000, 15002, 15003, 15004 and 15005 to redis. However when we try to conenct using 15001. We are getting below error

Failure case:

redis-cli -h our-redis-host.redis.cache.windows.net -p 15001 -a our-account-key --cacert "BaltimoreCyberTrustRoot.pem" --tls ping

OUTPUT:
Warning: Using a password with ā€˜-aā€™ or ā€˜-uā€™ option on the command line interface may not be safe.
Could not connect to Redis at our-redis-host.redis.cache.windows.net :15001: SSL_connect failed: Success

I could not see any entry in istio-proxy logs when trying from 15001 port. However when trying for other ports we can see entry in logs as below

[2021-05-05T00:59:18.677Z] "- - -" 0 - - - "-" 600 3982 10 - "-" "-" "-" "-" "172.XX.XX.XX:6380" PassthroughCluster 172.XX.XX.XX:45478 172.22.XX.XX:6380 172.XX.XX.XX:45476 - -

Is this because 15001 port blocks the outbound requests or manipulates certs for requests on 15001 port. If yes, is there any configuration to update the proxy_port to other ports than 15001?

1 Like
2

Hello @sravansagi, we faced the same issue and there is no one configuration that fixes this. As a workaround we did the following mesh configuration:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: istiocontrolplane
spec:
  meshConfig:
    proxyListenPort: 16001
    defaultConfig:
      proxyAdminPort: 16000

But this is not enough, you also need to update istio-sidecar-injector configmap to make sure iptables start forwarding traffic configured port. e.g

apiVersion: v1
kind: ConfigMap
metadata:
  name: istio-sidecar-injector
...
        args:
        - istio-iptables
        - "-p"
        - "16001"
...

It would be great if proxyListenPort also updates that port in istio-sidecar-injector configuraiont

1 Like