Currently I have k8s cluster 1.14.7 running on GKE and istio 1.0.7 installed with helm.
So I wanted to upgrade to latest release to get the cool feature and security updates. To do that I followed those step:
-
delete old crds istio 1.0.7
kubectl delete -f <old CRDs>
-
install istio-init 1.3.3
helm install gcs-repo/istio-init --namespace istio-system --name istio-init --tls
-
Delete helm chart to istio 1.3.3 because upgrading the chart will fail
helm del --purge <istio_release_name>
-
Install istio 1.3.3
helm install <istio_chart> --name istio --namespace istio-system
-
Rollout deployment to re inject istio sidecar
kubectl rollout restart deployment --namespace default
First I tried those steps on minikube everything works like a charm but once I want to upgrade my GKE cluster the Nginx is facing issues and logs the following:
E1104 13:26:12.246140 7 reflector.go:125] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:158: Failed to list *v1.ConfigMap: Get https://10.111.240.1:443/api/v1/configmaps?limit=500&resourceVersion=0: dial tcp 10.111.240.1:443: connect: connection refused
E1104 13:26:12.445845 7 reflector.go:125] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:155: Failed to list *v1.Endpoints: Get https://10.111.240.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.111.240.1:443: connect: connection refused
{ "time": "2019-11-04T13:26:12+00:00", "remote_addr": "146.148.11.246", "x-forward-for": "146.148.11.246", "request_id": "fdde30b4a97228da7a06c07c7c9ed3d1", "request": "GET /api/v1/<someendpoints> HTTP/1.1", "bytes_sent": 0, "request_time": 0.000, "status":502, "vhost": "api.mywebsite.io", "request_proto": "HTTP/1.1", "path": "/_external-auth-L2FwaS92MS90aGluZ3M","request_query": "-", "request_length": 0, "duration": 0.000, "method": "GET", "http_referrer": "-", "http_user_agent":"Go-http-client/1.1", "upstream_name":"default-myservice-http-80", "upstream_addr":"10.111.246.218:80","upstream_status":"502"}
{ "time": "2019-11-04T13:26:12+00:00", "remote_addr": "146.148.11.246", "x-forward-for": "146.148.11.246", "request_id": "fdde30b4a97228da7a06c07c7c9ed3d1", "request": "GET /api/
What am doing wrong here ? and why Nginx is unable to works ?
But when I rollback to istio 1.0.7 everything is back to normal.