Is there a way to configure a default (or are there already?) root CA certs for mTLS? The mTLS configuration takes the server key and cert and a cacert for validating clients. The cacert seems to be required, but what if the client certs only use already trusted CAs? Can we skip the intermediate cacerts and use default trusted CAs? Is there a way to configure a global list of trusted CAs?
Similar to Certificate Validation with SDS enabled on Ingress Gateway and mode set to MUTUAL
@JimmyChen who has reviewed https://github.com/istio/istio/pull/13363
That might help, but I’m thinking more of this: https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/auth/cert.proto
Where is the certificateValidationContext configured? Does the one in the mTLS section (related to 13363) augment or override the trusted_ca in the context? Is the trusted_ca in the context only used inside the mesh or is it also for ingress?
Hi @Phlak106 when you config a Gateway CRD and specify tls.mode to MUTUAL (example is here), Pilot will generate a certificateValidationContext config for ingress gateway which is Envoy.
That certificateValidationContext contains SDS config, so that Envoy will fetch ca cert from Gateway agent via SDS.
Assume you create a k8s secret named bookinfo-credential, which has key, server cert and cacert. Gateway agent will query API server for k8s secrets, set up a map, with key bookinfo-credential mapping to server key/cert, and key bookinfo-credential-cacert mapping to cacert.
Envoy will send SDS requests, one for server key/cert, and one for cacert. And the SDS request has a resource name is bookinfo-credential or bookinfo-credential-cacert. Gateway agent looks up the map and sends back the resources. Envoy uses the fetched key/cert and cacert to config a mTLS listener.
If you are using cert-manager, which only creates k8s secret with server key/cert. Then you have to manually create a generic k8s secret to wrap server key/cert and cacert (command).
Once https://github.com/istio/istio/pull/13363 is in, you can still use cert-manager to create k8s secret with key/cert, and manually create a separate k8s secret to store the cacert. I am not sure if Envoy already supports cacert list. If that is ready, we can modify the gateway agent to support it as well.
Wait, hold up, I am confused now.
IIRC the Gateway has nothing to do with Citadel. The Gateway’s proxy or Envoy container connects to the ingress-sds container, which implements an ``SDS server, over the secure Unix Domain Socket (UDS). Ingress-sds runs a SecretFetcher control loop which is essentially a ListWatcher for k8s Secrets configured in Gateway.Server.TLSOption.CredentialName.
Am I missing something @JimmyChen ?
@Phlak106 An alternate option to, Split the server/client secret fetching into two k8s secrets by farcaller · Pull Request #13363 · istio/istio · GitHub, would be configure the Secret’s cacert with a ca-certificate bundle (maybe with the contents of /etc/ssl/certs/ ca-certificates.crt). Not the best but a bundle might be the best way at the moment.
In my situation I created a helper controller that bridges this gap between cert-manager and istio-gateway. Essentially an adapter to convert cert-manager generated secrets to istio gateway recognized secret (type: generic) with an option to provide ca-certificates via a ConfigMap.
@nitishm Thanks for the update! Gateway has nothing to do with Citadel. I was trying to type Citadel agent, which is Gateway agent in the ingress SDS deployment. A controller that converts the secret is a good idea!
The basic design is shown below,
If it is something that could benefit the istio community I would be willing to contribute this controller (after discussions with my team) ?!
I will pick up https://github.com/istio/istio/pull/13363 this week. Let’s see if that solves the issue. If there are customers that still look for alternative solution. Your approach would be very helpful. @Oliver
I see that Pilot (https://github.com/istio/istio/blob/3dd9aa4dce25d3931670bdf467346ac7eb6867fb/pilot/pkg/networking/core/v1alpha3/gateway.go)
does create a CombinedValidationContext to configure Envoy with:
CombinedValidationContext: &auth.CommonTlsContext_CombinedCertificateValidationContext{
DefaultValidationContext: &auth.CertificateValidationContext{VerifySubjectAltName: server.Tls.SubjectAltNames},
ValidationContextSdsSecretConfig: authn_model.ConstructSdsSecretConfigForGatewayListener(
server.Tls.CredentialName+authn_model.IngressGatewaySdsCaSuffix, authn_model.IngressGatewaySdsUdsPath),
}
I’d like to propose exposing more of the default validation context through the api and plumbing it through here. I have submitted a small PR: https://github.com/istio/api/pull/969/files
This becomes mutually exclusive to SDS though right ? The DefaultValidationContext is not chosen if SDS is enabled. Am I understanding that correctly ?
My understanding of the Envoy documentation is that they will be merged:
Combined certificate validation context holds a default CertificateValidationContext and SDS config. When SDS server returns dynamic CertificateValidationContext, both dynamic and default CertificateValidationContext are merged into a new CertificateValidationContext for validation. This merge is done by Message::MergeFrom(), so dynamic CertificateValidationContext overwrites singular fields in default CertificateValidationContext, and concatenates repeated fields to default CertificateValidationContext, and logical OR is applied to boolean fields.
So the Gateway seems to be configuring Envoy the way I would expect, but just doesn’t expose the rest of the options to be able to be set in the default validation context. This would seem to allow setting a default CA cert in the default context and if SDS doesn’t overwrite it (through the merge) then it would be used as expected.
Any update @JimmyChen?
Now with the new incomig Istio release-1.3, the client root CA could be stored in a separate k8s secret. So that the rotation of client root CA is independent of the rotation of server key/cert. User guide is updated as well.