Hi, am trying to setup istio external control plane using 1.9 istio version referred istio 1.9 official doc for external control plane
Issue: my ingress pod is not working on remote cluster it’s say handshake failed
I have raised issue with details logs, please guide me
Thanks
opened 12:25PM - 08 Jun 21 UTC
feature/Multi-cluster
(**NOTE**: This is used to report product bugs:
To report a security vulnerab… ility, please visit <https://istio.io/about/security-vulnerabilities>
To ask questions about how to use Istio, please visit <https://discuss.istio.io>)
**Bug description**
i am trying to setup external control plane but istio-ingressgateway pod is not working on remote cluster under external-istiod namespaces
followed official doc https://istio.io/v1.9/docs/setup/additional-setup/external-controlplane/#requirements
pod status on external cluster
```
ubuntu@bastion-3f5f82d3~/istio-ankita/istio-1.9.5$ kubectl get pod -n istio-system
NAME READY STATUS RESTARTS AGE
httpbin-66cdbdb6c5-4mtsx 1/1 Running 0 4h35m
istio-ingressgateway-78d87ddc9b-xdkkr 1/1 Running 0 7h44m
istiod-cd489d56f-45wlt 1/1 Running 0 7h44m
ubuntu@bastion-3f5f82d3:~/istio-ankita/istio-1.9.5$ kubectl get pod -n external-istiod
NAME READY STATUS RESTARTS AGE
istiod-687bd4fc75-znjml 1/1 Running 0 170m
ubuntu@bastion-3f5f82d3:~/istio-ankita/istio-1.9.5$
```
but on remote its not working
```
ubuntu@bastion-3f5f82d3:~/istio-ankita/istio-1.9.5$ kubectl get pod -n external-istiod --kubeconfig config NAME READY STATUS RESTARTS AGE
istio-ingressgateway-65bdf697f5-tgxf8 0/1 Running 0 77m
```
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
**Expected behavior**
pod should be run and my deployment also but none working
**Steps to reproduce the bug**
followed official doc from istio.io on gcp cluster kuberenets v 1.18, istio v 1.9
**Version** (include the output of `istioctl version --remote` and `kubectl version --short` and `helm version --short` if you used Helm)
```
istio-1.9.5$ istioctl version --remote
client version: 1.9.5
control plane version: 1.9.5
data plane version: 1.9.5 (1 proxies)
istio-1.9.5$ istioctl version --remote
client version: 1.9.5
control plane version: 1.9.5
data plane version: 1.9.5 (1 proxies)
```
```
ubuntu@bastion-3f5f82d3:~/istio-ankita/istio-1.9.5$ kubectl version --short
Client Version: v1.21.1
Server Version: v1.18.17-gke.1900
```
heml not using
on external cluster :-
installed default profile
```
ubuntu@bastion-3f5f82d3:~/istio-ankita/istio-1.9.5$ istioctl version
client version: 1.9.5
control plane version: 1.9.5
data plane version: 1.9.5 (1 proxies)
```
logs from remote cluster pod
```
kubectl logs -f istio-ingressgateway-65bdf697f5-tgxf8 -n external-istiod --kubeconfig config
42726->34.136.114.249:15012: read: connection reset by peer"
2021-06-08T11:31:34.113133Z warn ca ca request failed, starting attempt 1 in 95.059667ms
2021-06-08T11:31:34.208445Z warn ca ca request failed, starting attempt 2 in 181.918749ms
2021-06-08T11:31:34.390653Z warn ca ca request failed, starting attempt 3 in 377.025674ms
2021-06-08T11:31:34.767973Z warn ca ca request failed, starting attempt 4 in 846.149233ms
2021-06-08T11:31:35.184167Z warn Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2021-06-08T11:31:35.614499Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: read tcp 10.48.0.5:42726->34.136.114.249:15012: read: connection reset by peer"
2021-06-08T11:31:35.851115Z warn ca ca request failed, starting attempt 1 in 108.643931ms
2021-06-08T11:31:35.960035Z warn ca ca request failed, starting attempt 2 in 199.209508ms
2021-06-08T11:31:36.159623Z warn ca ca request failed, starting attempt 3 in 379.215917ms
2021-06-08T11:31:36.539148Z warn ca ca request failed, starting attempt 4 in 745.637641ms
2021-06-08T11:31:37.184094Z warn Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2021-06-08T11:31:37.285197Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: read tcp 10.48.0.5:42726->34.136.114.249:15012: read: connection reset by peer"
2021-06-08T11:31:37.572627Z warn ca ca request failed, starting attempt 1 in 97.247705ms
2021-06-08T11:31:37.670102Z warn ca ca request failed, starting attempt 2 in 203.773252ms
2021-06-08T11:31:37.874270Z warn ca ca request failed, starting attempt 3 in 374.335894ms
2021-06-08T11:31:38.248949Z warn ca ca request failed, starting attempt 4 in 846.88025ms
2021-06-08T11:31:39.096360Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: read tcp 10.48.0.5:42726->34.136.114.249:15012: read: connection reset by peer"
2021-06-08T11:31:39.184199Z warn Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2021-06-08T11:31:39.475919Z warn ca ca request failed, starting attempt 1 in 93.574344ms
2021-06-08T11:31:39.569872Z warn ca ca request failed, starting attempt 2 in 183.407212ms
2021-06-08T11:31:39.753649Z warn ca ca request failed, starting attempt 3 in 416.775826ms
2021-06-08T11:31:40.170846Z warn ca ca request failed, starting attempt 4 in 772.378838ms
```
when i describe pod, i was below error
```
kubectl describe pod istio-ingressgateway-65bdf697f5-tgxf8 -n external-istiod --kubeconfig config
SizeLimit: <unset>
istio-data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
istio-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 43200
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: istio
Optional: true
ingressgateway-certs:
Type: Secret (a volume populated by a Secret)
SecretName: istio-ingressgateway-certs
Optional: true
ingressgateway-ca-certs:
Type: Secret (a volume populated by a Secret)
SecretName: istio-ingressgateway-ca-certs
Optional: true
istio-ingressgateway-service-account-token-x9xdd:
Type: Secret (a volume populated by a Secret)
SecretName: istio-ingressgateway-service-account-token-x9xdd
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 35s (x2686 over 90m) kubelet Readiness probe failed: HTTP probe failed with statuscode: 503
```
**How was Istio installed?**
istioctl install -f controlplane-gateway.yaml (as per doc)
**Environment where the bug was observed (cloud vendor, OS, etc)**
gcp cloud provider
Additionally, please consider running `istioctl bug-report` and attach the generated cluster-state tarball to this issue.
Refer [cluster state archive](http://istio.io/help/bugs/#generating-a-cluster-state-archive) for more details.
1 Like