Trying to integrating Istio with Azure Keyvault

1

Hi all,

I have an instance of K8S, the latest istio and the csi driver installed on my cluster. I want to configure an istio gateway to alllow http and https traffic.

Virtual network and ingress are configured, and right now the HTTP traffic flows just fine to my pod. But HTTPS traffic does not flow.

In azure, I have an A record pointing to the external ip of my ingress. I also have an app certificate configured for my domain. App certificate is connected to my keyvault; this connection creating a secret in my KV. I’m referencing the keyvault and this secret in my secretProviderClass YAML below.

This is my configuration:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: my-gateway
spec:
  # The selector matches the ingress gateway pod labels.
  # If you installed Istio using Helm following the standard documentation, this would be "istio=ingress"
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 8080
      name: http
      protocol: HTTP
    hosts:
    - "*"
  - port:
      number: 443
      name: https-443
      protocol: HTTPS
    tls:
      mode: SIMPLE # enables HTTPS on this port
      credentialName: ingress-cert-tls
    hosts:
    - '*'
---
apiVersion: v1
kind: Namespace
metadata:
  name: mynamespace
  labels:
    istio-injection: enabled
spec: {}
status: {}

---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  labels:
    app.kubernetes.io/managed-by: Helm
  name: myapp
  namespace: mynamespace
spec:
  gateways:
  - default/mygateway
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /apis/my-api
    route:
    - destination:
        host: mypodsvc
---
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: istio-tls
spec:
  provider: azure
  secretObjects:
    - secretName: ingress-cert-tls
      type: kubernetes.io/tls
      data: 
        - objectName: aks-ingress-cert # name of the mounted content to sync with K8s Secret object
          key: tls.key
        - objectName: aks-ingress-cert
          key: tls.crt
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: #MI_CLIENTID#
    keyvaultName: #KV_NAME#
    objects: |
      array:
        - |
          objectName: #SECRET_NAME_IN_THE_KEYVAULT#
          objectType: secret
    tenantId: #TENANT_ID#
---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
 components:
   pilot:
     enabled: true
     k8s:
       overlays:
         - kind: Deployment
           name: istiod
           patches:
             - path: spec.template.spec.volumes[name:cacerts]
               value:
                 name: cacerts
                 csi:
                   driver: secrets-store.csi.k8s.io
                   readOnly: true
                   volumeAttributes:
                     secretProviderClass: "istio-tls"
---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: ingress
spec:
  profile: empty # Do not install CRDs or the control plane
  components:
    ingressGateways:
    - name: istio-ingressgateway
      namespace: istio-system
      enabled: true
      label:
        # Set a unique label for the gateway. This is required to ensure Gateways
        # can select this workload
        istio: ingressgateway
  values:
    gateways:
      istio-ingressgateway:
        # Enable gateway injection
        injectionTemplate: gateway

What am I doing wrong?

P.s. I noticed that istiod won’t mount the volume if my secretproviderclass is not in the istio-system namespace. Is this expected?

2

I managed to sort the problem. There were a few problems in the templates, but the most important issue was caused by the certificate, that Azure App Certificate saves a secret in a keyvault instead of a certificate, so I had to export it and import it to make this work.

This is my full template, in case anybody needs a reference:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: my-gateway
  namespace: istio-system
spec:
  # The selector matches the ingress gateway pod labels.
  # If you installed Istio using Helm following the standard documentation, this would be "istio=ingress"
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 8080
      name: http
      protocol: HTTP
    hosts:
    - "*"
  - port:
      number: 443
      name: https-443
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: ingress-cert-tls
    hosts:
    - '*.domain.com'
---
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  labels:
    app.kubernetes.io/managed-by: Helm
  name: myapp
  namespace: mynamespace
spec:
  gateways:
  - istio-system/mygateway #refering the gateway in istio-system
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /apis/my-api
    route:
    - destination:
        host: mypodsvc
---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
 components:
   pilot:
     enabled: true
     k8s:
       overlays:
         - kind: Deployment
           name: istiod
           patches:
             - path: spec.template.spec.volumes[name:cacerts]
               value:
                 name: cacerts
                 csi:
                   driver: secrets-store.csi.k8s.io
                   readOnly: true
                   volumeAttributes:
                     secretProviderClass: "istio-tls"
---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: ingress
spec:
  profile: empty # Do not install CRDs or the control plane
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        overlays:
          - kind: Deployment
            name: istio-ingressgateway
            patches:
              - path: spec.template.spec.volumes[name:ingressgateway-certs]
                value:
                  name: ingressgateway-certs
                  csi:
                    driver: secrets-store.csi.k8s.io
                    readOnly: true
                    volumeAttributes:
                      secretProviderClass: "istio-tls"
      label:
        # Set a unique label for the gateway. This is required to ensure Gateways
        # can select this workload
        istio: ingressgateway
  values:
    gateways:
      istio-ingressgateway:
        # Enable gateway injection
        injectionTemplate: gateway
---
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: istio-tls
  namespace: istio-system
spec:
  provider: azure
  secretObjects:
    - secretName: ingress-cert-tls
      type: kubernetes.io/tls
      data: 
        - objectName: #CERT_NAME# # This is name of the certificate, not a secret!
          key: tls.key
        - objectName: #CERT_NAME#
          key: tls.crt
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: #MI_CLIENTID#
    keyvaultName: #KV_NAME#
    objects: |
      array:
        - |
          objectName: #CERT_NAME#
          objectType: secret
    tenantId: #TENANT_ID#