Hi, I’m trying to follow the instructions from your documentation:
But I’m getting the following error:
error: unable to recognize “STDIN”: no matches for kind “Certificate” in version “certmanager.k8s.io/v1alpha1”
Any help would be greatly appreciated.
Hi,
Istio installation contain older (0.6.2?) version of cert-manager. I think that you installed CRDs for the newer version.
It turns out the problem is not with the cert-manager version at all. I got the same error with version v0.6.2 as with the latest v0.13.0 of cert-manager. The problems are with the manifests that follow. I’ll try to figure it out and post what I come up with here.
It would appear the issue is with the api-group that the CRD is a member of. According to cert-manager documentation, you should use:
apiVersion: cert-manager.io/v1alpha2
This worked for me. Perhaps the documentation needs to be updated on Istio side.
With the latest version of cert-manager (0.13.0) I am now running into this open cert-manager issue where the ClusterIssuer is not found. https://github.com/jetstack/cert-manager/issues/2487
Maybe an older version of cert-manager will work properly, so maybe Tomas was right after all. Thanks for your assistance.
Hi,
we use EKS 1.14 and cert-manager 0.13.0 and everything works for us. 
Well, I got it to work once. I want to make sure I have all the steps right, so I deleted the cluster and tried again. Now, it doesn’t work, and I tried a few times and it’s a solid failure. I skipped the self-signed and staging issuers, so maybe something about creating those makes it work. I dunno. Below are all the steps, culminating with a 503 error. I’m using Digital Ocean.
# Digital Ocean Kubernetes v1.16.6 cluster - 4 nodes x 2vCPUs 4GB RAM each - total 6vCPUs 12GB RAM.
# (Set it to autoscale under "Nodes" to min 4, max 8 nodes just in case)
# For the commands below, omit the $. That's a bash prompt. I use Git Bash on Windows
# I'm using kubectl client v1.17.0
# Install Istio per the instructions here https://istio.io/docs/tasks/traffic-management/ingress/ingress-certmgr/
$ istioctl manifest apply \
--set values.gateways.istio-ingressgateway.sds.enabled=true \
--set values.global.k8sIngress.enabled=true \
--set values.global.k8sIngress.enableHttps=true \
--set values.global.k8sIngress.gatewayName=ingressgateway
# Wait until all Istio pods are ready...
$ kubectl get pods -n istio-system
# Wait a few minutes for the LoadBalancer which was created by the
# Istio install to be ready and show 3/3 pods.
# Create a DNS A record that points to the load balancer external IP - like host.example.com
# Set the $INGRESS_DOMAIN environment variable equal to the hostname of your cluster you
# created the DNS A record earlier.
$ INGRESS_DOMAIN=host.example.com
# Make sure you can resolve the host name by pinging the server...
# Install Cert-Manager 0.13.0
$ kubectl create namespace cert-manager
$ kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.13.0/cert-manager.yaml
# Wait for all 3 cert-manager pods to be running
$ kubectl get pods -n cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-6f578f4565-qc9mk 1/1 Running 0 14m
cert-manager-cainjector-75b6bc7b8b-wmtf4 1/1 Running 0 14m
cert-manager-webhook-8444c4bc77-d4bfq 1/1 Running 0 14m
# Patch the Istio Gateway per the instructions.
$ kubectl -n istio-system \
patch gateway istio-autogenerated-k8s-ingress --type=json \
-p='[{"op": "replace", "path": "/spec/servers/1/tls", "value": {"credentialName": "ingress-cert", "mode": "SIMPLE", "privateKey": "sds", "serverCertificate": "sds"}}]'
# Create the cluster Issuer. Update the email address to your email address.
$ cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
email: admin@example.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: istio-ingressgateway-certs
solvers:
- http01:
ingress:
class: istio
---
EOF
# Setup the helloworld application. Note this uses the $INGRESS_DOMAIN variable.
$ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
name: helloworld
labels:
app: helloworld
spec:
ports:
- port: 5000
name: http
selector:
app: helloworld
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: helloworld
spec:
selector:
matchLabels:
app: helloworld
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: helloworld
image: istio/examples-helloworld-v1
resources:
requests:
cpu: "100m"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5000
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: istio
name: helloworld-ingress
spec:
rules:
- host: "$INGRESS_DOMAIN"
http:
paths:
- path: /hello
backend:
serviceName: helloworld
servicePort: 5000
---
EOF
# Wait for helloworld to be ready
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
helloworld-6b489b9859-bvxcw 1/1 Running 0 6m10s
# Confirm that hello world is working non-https:
$ curl http://$INGRESS_DOMAIN/hello
Hello version: v1, instance: helloworld-6b489b9859-9pqc5
# Finally, create the certificate.
$ cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: ingress-cert
namespace: istio-system
spec:
secretName: ingress-cert
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
commonName: 'host.example.com'
dnsNames:
- 'host.example.com'
---
EOF
# Wait for the certificate to be ready:
$ kubectl describe certificate ingress-cert -n istio-system
# Wait for the Certificate Request (the cert id comes from the output of the above command)
$ kubectl describe certificaterequest ingress-cert-4051514424 -n istio-system
# Wait for the Certificate Order (The order id comes from output of the above command)
$ kubectl describe order ingress-cert-4051514424-3229718444 -n istio-system
# Wait for the challenge (the challenge id comes from the output of the above command)
$kubectl describe challenge ingress-cert-4051514424-3229718444-3565931193 -n istio-system
Reason: Waiting for http-01 challenge propagation: wrong status code '503', expected '200'
State: pendingOk, I found a reference to someone having the same problem here https://community.letsencrypt.org/t/error-503-using-cert-manager/107930/3 , and they solved it by deleting and re-creating the ingress. I tried this and it worked for me too. Not clear why this is required, but it is. Note that just deleting the pod and letting Kubernetes recreate it for you doesn’t solve the issue. You have to do “kubectl delete -f ingress.yml”, and then “kubectl apply -f ingress.yml” to solve the problem.
Hi,
Wanted to ask how you got the ingress to serve outside traffic?
I’m having issues with the ingress part at https://istio.io/docs/tasks/traffic-management/ingress/ingress-certmgr/#setting-up-a-demo-application
I’m installing as described, using k8s 1.17 with istio 1.4.4, NodePort type, with no success.
I’m able to get traffic using Gateway and VirtualService resources, but not using regular kubernetes ingress resource.
getting Connection refused on port 80.
Other ingresses work fine (nginx traefik etc…), so what am i missing regarding the kubernetes.io/ingress.class: istio annotation?
I think your question is sufficiently unique that it warrants a new thread on its own. I can see why you might not want to pay extra for a Load Balancer. If it can be done via NodePort safely, I don’t want to pay extra either. The LoadBalancer from the example above was created automatically at the first step, so I went with it. This issue, with Cert-Manager took me about 3 days of trial, error and investigation to resolve. In the process, I did come across some references that might help you.
From here: https://istio.io/docs/setup/getting-started/
“If your cluster is running in an environment that does not support an external load balancer (e.g., minikube), the EXTERNAL-IP of istio-ingressgateway will say <pending> . To access the gateway, use the service’s NodePort , or use port-forwarding instead.”
In the above reference to “the service” I assume the author must mean the “istio-ingressgateway”. I also assume that by “use” they mean “expose”, which you can find instructions for here:
https://kubernetes.io/docs/tutorials/stateless-application/expose-external-ip-address/
Which lists the command:
kubectl expose deployment hello-world --type=LoadBalancer --name=my-service
Where of course you would use NodePort instead of LoadBalancer and istio-ingressgateway as required for the other two parameters, not hello-world and/or my-service, although I’m not exactly sure which here.
Good luck in your journey.
Just in case I didn’t answer your question directly enough… The load balance was created automatically when I installed Istio. A few minutes after you kick-off the Istio installation, the external address will appear, and it will show that it is balancing requests to all the nodes in your cluster. I don’t think Istio supports using a Node Port for the istio-ingressgateway service, someone from the Istio team might be able to comment on that.
To get the external IP of the Load Balancer, you can just look in the Digital Ocean interface at the definition of the Load Balancer. Again, it takes a few minutes to appear after it is created. You can also run the following command and look at the External IP:
From: https://istio.io/docs/tasks/traffic-management/ingress/ingress-certmgr/
kubectl -n istio-system get service istio-ingressgateway
Hi, I found that if you create the Ingress last, there is no need to delete and re-create it. The complete, working instructions are below.
# Digital Ocean Kubernetes v1.16.6 cluster - 3 nodes x 2vCPUs 4GB RAM each - total 6vCPUs 12GB RAM.
# (Set it to autoscale under "Nodes" to min 3, max 8 nodes just in case)
# Install Istio per the instructions here https://istio.io/docs/tasks/traffic-management/ingress/ingress-certmgr/
$ istioctl manifest apply \
--set values.gateways.istio-ingressgateway.sds.enabled=true \
--set values.global.k8sIngress.enabled=true \
--set values.global.k8sIngress.enableHttps=true \
--set values.global.k8sIngress.gatewayName=ingressgateway
# Wait until all Istio pods are ready...
$ kubectl get pods -n istio-system
# Wait a few minutes for the LoadBalancer which was created by the
# Istio install to be ready and show 3/3 pods.
# Create a DNS A record that points to the load balancer external IP - like host.example.com
# Set the $INGRESS_DOMAIN environment variable equal to the hostname of your cluster you
# created the DNS A record earlier.
$ INGRESS_DOMAIN=host.example.com
# Install Cert-Manager 0.13.0
$ kubectl create namespace cert-manager
$ kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.13.0/cert-manager.yaml
# Wait for all 3 cert-manager pods to be running
$ kubectl get pods -n cert-manager
# Patch the Istio Gateway per the instructions.
$ kubectl -n istio-system \
patch gateway istio-autogenerated-k8s-ingress --type=json \
-p='[{"op": "replace", "path": "/spec/servers/1/tls", "value": {"credentialName": "ingress-cert", "mode": "SIMPLE", "privateKey": "sds", "serverCertificate": "sds"}}]'
# Now RESTART ISTIO and CERT-MANAGER or else they won't be able to resolve your new DNS entry.
# First, restart Istio
$ kubectl delete pods --all -n istio-system
# Wait and confirm that Istio is all up and running
$ kubectl get pods -n istio-system
# Restart Cert Manager
$ kubectl delete pods --all -n cert-manager
# Wait until Cert Manager is all back up and running again
$ kubectl get pods -n cert-manager
# Setup the helloworld application. Note this uses the $INGRESS_DOMAIN variable.
$ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
name: helloworld
labels:
app: helloworld
spec:
ports:
- port: 5000
name: http
selector:
app: helloworld
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: helloworld
spec:
selector:
matchLabels:
app: helloworld
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: helloworld
image: istio/examples-helloworld-v1
resources:
requests:
cpu: "100m"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5000
---
EOF
# Create the Cluster Issuer. Update the email address to your email address.
$ cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
email: admin@example.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: istio-ingressgateway-certs
solvers:
- http01:
ingress:
class: istio
---
EOF
# Wait for helloworld to be ready
$ kubectl get pods
# Create the certificate.
$ cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: ingress-cert
namespace: istio-system
spec:
secretName: ingress-cert
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
commonName: 'host.example.com'
dnsNames:
- 'host.example.com'
---
EOF
# Create the Ingress.
$ cat <<EOF | kubectl apply -f -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: istio
name: helloworld-ingress
spec:
rules:
- host: "$INGRESS_DOMAIN"
http:
paths:
- path: /hello
backend:
serviceName: helloworld
servicePort: 5000
---
EOF
# Wait for the certificate to be ready:
$ kubectl -n istio-system describe certificate ingress-cert
# Wait for the Certificate Request (the cert id comes from the output of the above command)
$ kubectl -n istio-system describe certificaterequest ingress-cert-4051514424
# Wait for the Certificate Order (The order id comes from output of the above command)
$ kubectl -n istio-system describe order ingress-cert-4051514424-3229718444
# Wait for the challenge (the challenge id comes from the output of the above command)
$ kubectl -n istio-system describe challenge ingress-cert-4051514424-3229718444-3565931193Hi Chrisnyc
Could you please renew the doc with your steps, because https://istio.io/docs/tasks/traffic-management/ingress/ingress-certmgr/ does not work at all.
Thanks
Hi, sorry I do not work for the Istio team. I don’t have access to update the document. I’m just like you. Someone who found that the steps in the documentation didn’t work, so I tried different things until I could make it work.
Hello Chris!
Thank you for detailing out the correct instructions. I have a question that I was hoping you can help me out with. Apologies in advance that this questions might come across as a rookie one.
If I were to send traffic from outside of EKS to the service within EKS, how do I authenticate the request?
After following your instructions, the problem seemed to have been solved but I am not sure how I can authenticate the request.
Hi, I’m mostly a Java programmer, so I went through this entire book, which covers authentication.
There are much simpler examples available, seems using NGINX is simplest. I haven’t tried this but it looks pretty simple. https://kubernetes.github.io/ingress-nginx/examples/auth/basic/
Best of luck to you in your journey.