Need Help as Istio is blocking the traffic using Helm Deployments with AWS EKS

Networking
1

Setup

  • AWS EKS K8s 1.8 Cluster (setup done through eksctl)

  • Nodes are Ubuntu Containers OS

  • Istio 1.9 installed through istioctl using the below command,

  • istioctl install --set profile=default --set meshConfig.outboundTrafficPolicy.mode=ALLOW_ANY

  • I have the Rancher deployment of application which has a pre-hook which will connect a Swarm cluster running outside the K8s Cluster on an AWS VM through anisbile playbook. (Ansible playbook is invoked in pre-install hook of our Helm chart creating the Oracle DB on the swarm nodes)

Now the Problem Statement,
Without Istio namespace Injection, everything works fine. But with istio injection, I am getting the below error saying connection refused. (I see that istio_proxy is getting created as sidecar)

> time="2021-03-02T16:00:02Z" level=error msg="Get \"https://172.20.0.1:443/api/v1/namespaces/default/configmaps/cluster-info\": dial tcp 172.20.0.1:443: connect: connection refused"
> Get "https://172.20.0.1:443/api/v1/namespaces/default/configmaps/cluster-info": dial tcp 172.20.0.1:443: connect: connection refused
> time="2021-03-02T16:00:02Z" level=info msg="Configmap already exists in the namespace \n"
> Get "https://172.20.0.1:443/api/v1/namespaces/default/configmaps/cluster-info": dial tcp 172.20.0.1:443: connect: connection refused
> time="2021-03-02T16:00:02Z" level=info
> exit status 1
> time="2021-03-02T16:00:02Z" level=debug msg="Command to run" Command object:="/ansible/bin/ansible-playbook --inventory playbook/inventory/hosts --extra-vars {\"servicename\":\"nj-istio\"} --connection ssh playbook/deploy-oracle.yaml"
> time="2021-03-02T16:00:02Z" level=debug msg="Running go routine to stream playbook execution result"
> [WARNING]: Unhandled error in Python interpreter discovery for host
> rndk8s19cdb.metricstream.com: Failed to connect to the host via ssh: ssh:
> connect to host rndk8s19cdb.metricstream.com port 22: Connection refused
> time="2021-03-02T16:00:03Z" level=error msg="Error Occured while running the playbook" error:="exit status 4"

====================

Tried with 1.8 , 1.9.1 as well. Not sure why istio is blocking the same

2

Looks like the issue here was,
Application Pod was trying to start and connect using K8s cluster thing before the istio proxy would have finished.
So as of now added the sleep in the application logic and the issue got solved
Also make use of the annotation to achieve the same
annotations:
proxy.istio.io/config: ‘{ “holdApplicationUntilProxyStarts”: true }’