Istio-ingressgateway Deployment to DaemonSet

Is it possible to change Deployment to DaemonSet for istio-ingressgateway? If I bruteforcly change manifest to DaemonSet it starts to complain about istio-token. Is it possible and if yes then how?

Thank you

apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    app: istio-ingressgateway
    install.operator.istio.io/owning-resource: unknown
    istio: ingressgateway
    istio.io/rev: default
    operator.istio.io/component: IngressGateways
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
      istio: ingressgateway
  template:
    metadata:
      annotations:
        prometheus.io/path: /stats/prometheus
        prometheus.io/port: "15020"
        prometheus.io/scrape: "true"
        sidecar.istio.io/inject: "false"
      labels:
        app: istio-ingressgateway
        chart: gateways
        heritage: Tiller
        install.operator.istio.io/owning-resource: unknown
        istio: ingressgateway
        istio.io/rev: default
        operator.istio.io/component: IngressGateways
        release: istio
        service.istio.io/canonical-name: istio-ingressgateway
        service.istio.io/canonical-revision: latest
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - preference:
                matchExpressions:
                  - key: kubernetes.io/arch
                    operator: In
                    values:
                      - amd64
              weight: 2
            - preference:
                matchExpressions:
                  - key: kubernetes.io/arch
                    operator: In
                    values:
                      - ppc64le
              weight: 2
            - preference:
                matchExpressions:
                  - key: kubernetes.io/arch
                    operator: In
                    values:
                      - s390x
              weight: 2
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: kubernetes.io/arch
                    operator: In
                    values:
                      - amd64
                      - ppc64le
                      - s390x
      containers:
        - args:
            - proxy
            - router
            - --domain
            - $(POD_NAMESPACE).svc.cluster.local
            - --proxyLogLevel=warning
            - --proxyComponentLogLevel=misc:error
            - --log_output_level=default:info
            - --serviceCluster
            - istio-ingressgateway
          env:
            - name: JWT_POLICY
              value: third-party-jwt
            - name: PILOT_CERT_PROVIDER
              value: istiod
            - name: CA_ADDR
              value: istiod.istio-system.svc:15012
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.nodeName
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: INSTANCE_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.podIP
            - name: HOST_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.hostIP
            - name: SERVICE_ACCOUNT
              valueFrom:
                fieldRef:
                  fieldPath: spec.serviceAccountName
            - name: CANONICAL_SERVICE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.labels['service.istio.io/canonical-name']
            - name: CANONICAL_REVISION
              valueFrom:
                fieldRef:
                  fieldPath: metadata.labels['service.istio.io/canonical-revision']
            - name: ISTIO_META_WORKLOAD_NAME
              value: istio-ingressgateway
            - name: ISTIO_META_OWNER
              value: kubernetes://apis/apps/v1/namespaces/istio-system/deployments/istio-ingressgateway
            - name: ISTIO_META_ROUTER_MODE
              value: standard
            - name: ISTIO_META_CLUSTER_ID
              value: Kubernetes
          image: docker.io/istio/proxyv2:1.8.1
          name: istio-proxy
          ports:
            - containerPort: 15021
              protocol: TCP
            - containerPort: 8080
              protocol: TCP
            - containerPort: 8443
              protocol: TCP
            - containerPort: 15012
              protocol: TCP
            - containerPort: 15443
              protocol: TCP
            - containerPort: 15090
              name: http-envoy-prom
              protocol: TCP
          readinessProbe:
            failureThreshold: 30
            httpGet:
              path: /healthz/ready
              port: 15021
              scheme: HTTP
            initialDelaySeconds: 1
            periodSeconds: 2
            successThreshold: 1
            timeoutSeconds: 1
          resources:
            limits:
              cpu: 2000m
              memory: 1024Mi
            requests:
              cpu: 100m
              memory: 128Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            privileged: false
            readOnlyRootFilesystem: true
          volumeMounts:
            - mountPath: /etc/istio/proxy
              name: istio-envoy
            - mountPath: /etc/istio/config
              name: config-volume
            - mountPath: /var/run/secrets/istio
              name: istiod-ca-cert
            - mountPath: /var/run/secrets/tokens
              name: istio-token
              readOnly: true
            - mountPath: /var/run/ingress_gateway
              name: gatewaysdsudspath
            - mountPath: /var/lib/istio/data
              name: istio-data
            - mountPath: /etc/istio/pod
              name: podinfo
            - mountPath: /etc/istio/ingressgateway-certs
              name: ingressgateway-certs
              readOnly: true
            - mountPath: /etc/istio/ingressgateway-ca-certs
              name: ingressgateway-ca-certs
              readOnly: true
      securityContext:
        fsGroup: 1337
        runAsGroup: 1337
        runAsNonRoot: true
        runAsUser: 1337
      serviceAccountName: istio-ingressgateway-service-account
      volumes:
        - configMap:
            name: istio-ca-root-cert
          name: istiod-ca-cert
        - downwardAPI:
            items:
              - fieldRef:
                  fieldPath: metadata.labels
                path: labels
              - fieldRef:
                  fieldPath: metadata.annotations
                path: annotations
          name: podinfo
        - emptyDir: {}
          name: istio-envoy
        - emptyDir: {}
          name: gatewaysdsudspath
        - emptyDir: {}
          name: istio-data
        - name: istio-token
          projected:
            sources:
              - serviceAccountToken:
                  audience: istio-ca
                  expirationSeconds: 43200
                  path: istio-token
        - configMap:
            name: istio
            optional: true
          name: config-volume
        - name: ingressgateway-certs
          secret:
            optional: true
            secretName: istio-ingressgateway-certs
        - name: ingressgateway-ca-certs
          secret:
            optional: true
            secretName: istio-ingressgateway-ca-certs
1 Like

In case somebody else is looking for a solution:
In my case, it was best to use IstioOperator:

and use Deployment strategy as DaemonSet is not able to restart without downtime.

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: gateways
  namespace: istio-system
spec:
  profile: empty
  components:
    base:
      enabled: true
      
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
        k8s:
          priorityClassName: system-node-critical
          resources:
            limits:
              memory: 5Gi
              
          strategy:
            type: RollingUpdate
            rollingUpdate: 
              maxSurge: 1
              maxUnavailable: 0
              
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: node-role.kubernetes.io/master
                        operator: DoesNotExist
          overlays:
            - kind: Deployment
              name: istio-ingressgateway
              patches:
                - path: spec.template.spec.topologySpreadConstraints
                  value: 
                    - maxSkew: 1
                      topologyKey: kubernetes.io/hostname
                      whenUnsatisfiable: ScheduleAnyway
                      labelSelector:
                        matchLabels:
                          istio: ingressgateway
              
          service:
            type: NodePort
            externalTrafficPolicy: Local
            selector:
              app: istio-ingressgateway          
            ports:
              - name: http
                port: 8000
                targetPort: 8000
                nodePort: 80
              - name: https
                port: 1443
                targetPort: 1443
                nodePort: 443