Custom CA on istio 1.6.2

Hi, I’ve been trying to get the replicated control plane to work on Istio 1.6.2 with a custom CA by following the guide located at Some details on what I’ve done:

Cluster: Google Kubernetes Engine
Type: Private Cluster

  1. Deployed a private CA using
  2. Generated a certificate, exported the root and intermediate certificates for the CA
  3. Validated that all certificates can be linked up their chain (actually, even used it for a hello-world webhost just to be absolutely sure they work)
  4. Renamed all the files to the correct naming and created cacerts: ca-cert.pem, ca-key.pem, cert-chain.pem and root-cert.pem
  5. Tried to deploy istio using the manifests/examples/multicluster/values-istio-multicluster-gateways.yaml as per the guide

However, istiod continuously fails to work with the certificate and reports the following error:
Error: failed to create discovery service: failed to create CA: failed to create an istiod CA: certificate is not authorized to sign other certificates

I’ve tried swapping certificates to a legitimate wildcard certificate from GoDaddy but that failed as well. If I delete cacerts or use Istio’s sample certs, the deployment works. I did some digging and found that the sample ca-cert.pem provided with istioctl has the following which my custom CA cert and GoDaddy certificate does not have:

X509v3 Key Usage:
Certificate Sign

Could someone verify if this is a required condition for certificates to work with istiod? Is there an easier way to go about getting a custom CA to work with 1.6.2? My apologies if I missed this in the documentation somewhere, but if anyone could point it out to me, it would help alot, thanks!

I’ve resolved this, here is what I did, hope it helps someone:

  1. Switched to cfssl as the CA, you need the ability to modify key usage and extended key usage of the certs
  2. Generated a CA Cert (root-cert.pem) & Intermediate Cert according to this guide:
  3. Generated a Cert for Istio’s usage (ca-cert.pem & ca-key.pem). This was the extremely tricky part. The cert should have the Key Usage “Certificate Signing” and if you are using the same cfssl version that I am, add the Extended Key Usage “any”*
  4. Combine the above 3 certs into a cert-chain.pem file
  5. Use the 4 files to deploy cacerts as per the Istio guide

*I lack the expertise to explain the necessity behind the “any” Extended Key Usage but I got the clue from here: on jessedearing’s comment about RFC 5280. If any Extended Key Usage exists, it must conform to the exact usage of the Key. Therefore, without knowing more about Istio’s usage of the Key, I added the “any” value. Please note, this could be insecure.


try generating root ca, intermediate ca, and cert chain using step certificate, it is more easier.

you can do something like:

step certificate create zufar-root-ca root-cert.pem root-key.pem --profile root-ca  --kty RSA --no-password --insecure --not-after 87600h --san
step certificate create zufar-intermediate-ca ca-cert.pem ca-key.pem --profile intermediate-ca --kty RSA --ca ./root-cert.pem --ca-key ./root-key.pem --no-password --insecure --not-after 43800h --san
step certificate bundle ca-cert.pem root-cert.pem cert-chain.pem

Hey Zufar,

Thanks for the reply, I actually tried smallstep first, but I could not modify the Key Usage and Key Extended Usage values.

I believe this is tracked here:

I also used cfssl to create certs and add in istio, now I am not getting this error: “certificate is not authorized to sign other certificates”, but I see this error: “grpc: Server.Serve failed to complete security handshake tls: unknown certificate authority”
Were you able to solve this?

I have a modified version of Makefile to create self signed certs

Follow the commands in comment to create self signed certs. I have tested this in 1.4 and 1.5 istio versions.

# Create self signed root and intermediate certs for testing etc.
# export # this is Org in intermediate ssl cert
# export # this is Org for root cert
# make intermediate-cluster1-certs # intermediate-cluster1 will be the Location in intermediate cert
# cd intermediate-cluster1
# kubectl create secret generic cacerts -n istio-system --from-file=ca-cert.pem --from-file=ca-key.pem --from-file=root-cert.pem --from-file=cert-chain.pem

.SUFFIXES: .csr .pem .conf
.PRECIOUS: %/ca-key.pem %/ca-cert.pem %/cert-chain.pem
.PRECIOUS: root-cert.csr root-ca.conf %/cluster-ca.csr %/intermediate.conf
.SECONDARY: root-cert.csr root-ca.conf %/cluster-ca.csr %/intermediate.conf


# variables: root CA
# Additional variables are defined in root-ca.conf target below.

# variables: intermediate CA (Citadel)
CITADEL_SERIAL ?= $(shell echo $$PPID) 	# certificate serial number (uses current PID)
CITADEL_CN ?= Intermediate CA
CITADEL_SAN_DNS ?= localhost
# Additional variables are defined in %/intermediate.conf target below.

##help:		print this help message
.PHONY: help

help: Makefile
	@sed -n 's/^##//p' $<

##root-ca:	generate root CA files (key and certifcate) in current directory
.PHONY: root-ca

root-ca: root-key.pem root-cert.pem

root-cert.pem: root-cert.csr root-key.pem
	@echo "generating $@"
	@openssl x509 -req -days $(ROOTCA_DAYS) -signkey root-key.pem \
		-extensions req_ext -extfile root-ca.conf \
		-in $< -out $@

root-cert.csr: root-key.pem root-ca.conf
	@echo "generating $@"
	@openssl req -new -key $< -config root-ca.conf -out $@ 

	@echo "[ req ]" > $@
	@echo "encrypt_key = no" >> $@
	@echo "prompt = no" >> $@
	@echo "utf8 = yes" >> $@
	@echo "default_md = sha256" >> $@
	@echo "default_bits = $(ROOTCA_KEYSZ)" >> $@
	@echo "req_extensions = req_ext" >> $@
	@echo "x509_extensions = req_ext" >> $@
	@echo "distinguished_name = req_dn" >> $@
	@echo "[ req_ext ]" >> $@
	@echo "subjectKeyIdentifier = hash" >> $@
	@echo "basicConstraints = critical, CA:true" >> $@
	@echo "keyUsage = critical, digitalSignature, nonRepudiation, keyEncipherment, keyCertSign" >> $@
	@echo "[ req_dn ]" >> $@
	@echo "O = $(ROOTCA_ORG)" >> $@
	@echo "CN = $(ROOTCA_CN)" >> $@

	@echo "generating $@"
	@openssl genrsa -out $@ 4096

##<name>-certs:	generate Citadel certificates for <name>. Includes all PEM files needed.
.PHONY: %-certs

%-certs: %/cert-chain.pem root-cert.pem
	@echo "Citadel inputs stored in $(dir $<)"
	@cp root-cert.pem $(dir $<)

%/cert-chain.pem: %/ca-cert.pem root-cert.pem
	@echo "generating $@"
	@cat $^ > $@

%/ca-cert.pem: %/cluster-ca.csr root-key.pem root-cert.pem
	@echo "generating $@"
	@openssl x509 -req -days $(CITADEL_DAYS) \
		-CA root-cert.pem -CAkey root-key.pem -set_serial $(CITADEL_SERIAL) \
		-extensions req_ext -extfile $(dir $<)/intermediate.conf \
		-in $< -out $@

%/cluster-ca.csr: L=$(dir $@)
%/cluster-ca.csr: %/ca-key.pem %/intermediate.conf
	@echo "generating $@"
	@openssl req -new -config $(L)/intermediate.conf -key $< -out $@ 

	@echo "generating $@"
	@mkdir -p $(dir $@)
	@openssl genrsa -out $@ 4096

%/intermediate.conf: L=$(dir $@)
	@echo "[ req ]" > $@
	@echo "encrypt_key = no" >> $@
	@echo "prompt = no" >> $@
	@echo "utf8 = yes" >> $@
	@echo "default_md = sha256" >> $@
	@echo "default_bits = $(CITADEL_KEYSZ)" >> $@
	@echo "req_extensions = req_ext" >> $@
	@echo "x509_extensions = req_ext" >> $@
	@echo "distinguished_name = req_dn" >> $@
	@echo "[ req_ext ]" >> $@
	@echo "subjectKeyIdentifier = hash" >> $@
	@echo "basicConstraints = critical, CA:true, pathlen:0" >> $@
	@echo "keyUsage = critical, digitalSignature, nonRepudiation, keyEncipherment, keyCertSign" >> $@
	@echo "subjectAltName=@san" >> $@
	@echo "[ san ]" >> $@
	@echo "URI.1 = spiffe://cluster.local/ns/istio-system/sa/citadel" >> $@
	@echo "URI.2 = spiffe://$(L:/=)/ns/istio-system/sa/citadel" >> $@
	@echo "DNS.1 = $(CITADEL_SAN_DNS)" >> $@
	@echo "[ req_dn ]" >> $@
	@echo "O = $(CITADEL_ORG)" >> $@
	@echo "CN = $(L:/=)" >> $@
	@echo "L = $(L:/=)" >> $@