Trying to Install Multi-Primary but fails at creating remote secret

1

Hoping someone ran into the same issue and can provide some further insight and guidance into the problem I am having.

$ istioctl x create-remote-secret \
>     --context="${CTX_CLUSTER1}" \
>     --name=cluster1 | \
>     kubectl apply -f - --context="${CTX_CLUSTER2}"
error: could not get access token to read resources from local kube-apiserver: wrong number of secrets (2) in serviceaccount istio-system/istio-reader-service-account
Error: could not get access token to read resources from local kube-apiserver: wrong number of secrets (2) in serviceaccount istio-system/istio-reader-service-account
error: no objects passed to apply
2

I JUST hit this very same problem a couple days ago. It is fixed already (PR merged) but is NOT in any released istioctl binary yet. You have to build your own.

You will get this if, for example, you are on OpenShift.

See this for how you can build your own istioctl to work around this: [hack] multi-cluster setup scripts by jmazzitelli · Pull Request #3713 · kiali/kiali · GitHub

Because istio/istio#30565 isn't released yet, to use this stuff, you need to build your own istioctl binary. So pull the latest istio master and run this - the resultant /tmp/istiobuild/istioctl should be copied into your Istio installation:

git checkout tags/1.9.2 -b 192-plus-istioctl-fix
git cherry-pick 1acfc29db93de5d46337c7f182b0c28456e4dcd7
make -e TARGET_OUT=/tmp/istiobuild gen-charts istioctl
cp /tmp/istiobuild/istioctl <where your existing istioctl is>
3

NOTE! If you use this istioctl to install things, you have to remember to pass in --set tag=1.9.2 because the tag default hardcoded in istioctl will be “1.9-dev” and those images aren’t on gcr.io - you need to tell the Istio Operator to use a released tag – 1.9.2 for example.

4

Thanks @jmazzitelli. Will give it a try.

5

This command doesn’t generate istioctl in the /tmp/istiobuild.

cp /tmp/istiobuild/istioctl .                                                          192-plus-istioctl-fix
cp: cannot stat '/tmp/istiobuild/istioctl': No such file or directory
6

@jmazzitelli
Tried:
make -e TARGET_OUT=/tmp/istiobuild gen-charts istioctl
on another computer and it worked there.
copied that istioctl to the existing istioctl.

Run into this problem after cleaning out the previous istio install and reinstall istio with the --set tag=1.9.2
Seems to have gotten just a litte further.

istioctl x create-remote-secret --kubeconfig=config-demo \
    --context="${CTX_CLUSTER1}" \
    --name=cluster1 | \
    kubectl apply -f - --context="${CTX_CLUSTER2}" --kubeconfig=config-demo
error: could not get access token to read resources from local kube-apiserver: wrong number of secrets (2) in serviceaccount istio-system/istio-reader-service-account, please use --secret-name to specify one
Error: could not get access token to read resources from local kube-apiserver: wrong number of secrets (2) in serviceaccount istio-system/istio-reader-service-account, please use --secret-name to specify one
error: no objects passed to apply
7

Error: could not get access token to read resources from local kube-apiserver: wrong number of secrets (2) in serviceaccount istio-system/istio-reader-service-account, please use --secret-name to specify one

Did you do what that error messages says to do?

8

Which is the correct one to pick? How do you determine that? Yes, this is happening in openshift.

9

I just picked the one with “token” in the name. Works for me.

10

But there is two with “token” in the name , perhaps that is why the program throws the error. Is it expecting only one?

11

It’s not like you have 100s of them, right? Should only be like 2. Pick one. If it doesn’t work, pick the other :blush: I bet you could use any of them - I just picked one of them to use and it worked for me.

Read the PR that fixes this issue for further details on why it (erroneously) expects just one: Introduce remote secret name parameter for "istioctl x create-remote-secret" by libesz · Pull Request #30565 · istio/istio · GitHub

12

Thanks. I picked one and that seems to work.
secret/istio-remote-secret-cluster1 created

However, the second part,
Install a remote secret in cluster1 that provides access to cluster2’s API server.
fails:

error: no "ca.crt" data found
Error: no "ca.crt" data found
error: no objects passed to apply

I tried both tokens there with the same results.

Following the instructions laid out here:
https://istio.io/latest/docs/setup/install/multicluster/multi-primary/

13

Feel free to check out the scripts I’m writing - might help you figure it out. These work for me (they work for both openshift and minikube). See the scripts in this PR: [hack] multi-cluster setup scripts by jmazzitelli · Pull Request #3713 · kiali/kiali · GitHub

14

Thank you. They helped and I was able to get further along. Much obliged.