Possibility to add custom labels

Config
1

Hi, we are trying to tidy out our Kubernetes cluster where we have Istio installed (version 1.6-dev). We would like to use standard and custom labels on everything running there. Unfortunately we are not able to set custom labels for Istio components that were installed through IstioOperator.

Current config:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  namespace: {{.Values.istioNamespace}}
  name: istio-outsystems
  labels:
    app.kubernetes.io/name: istio-system
    app.kubernetes.io/component: istio-operator
    app.kubernetes.io/part-of: istio
spec:
  profile: minimal
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
  addonComponents:
    prometheus:
      enabled: true
      k8s:
        replicaCount: 1
    kiali:
      enabled: true
      k8s:
        replicaCount: 1
    tracing:
      enabled: true
      k8s:
        replicaCount: 1
  values:
    global:
      defaultNodeSelector:
        "beta.kubernetes.io/os": linux
        "node/role": infra
    kiali:
      createDemoSecret: true

Current state:

app=jaeger,pod-template-hash=78c5df8c46,release=istio
app=istiod,istio.io/rev=default,istio=pilot,pod-template-hash=5ddb875c68
app=kiali,pod-template-hash=6887ddb445,release=istio
app=prometheus,pod-template-hash=65dfdd46c4,release=istio

As you can see, components installed through IstioOperator doesn’t have tags we would like to see there. I haven’t found anything related to this in documentation. I just want to be sure that there is no way to set this up before opening feature request to add option to specify custom labels.

2

AFAIK, you have to patch each and every object with overlays (kind of find and replace), here is an example for patching prometheus and kiali, use similar config for other objects

spec:
  profile: minimal
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
  addonComponents:
    prometheus:
      enabled: true
      k8s:
        overlays:
        - apiVersion: apps/v1
          kind: Deployment
          name: prometheus
          patches:
          - path: metadata.labels.custom-label
            value:
              deployment-custom-label-value
        replicaCount: 1
    kiali:
      enabled: true
      k8s:
        replicaCount: 1
        overlays:
        - apiVersion: apps/v1
          kind: Deployment
          name: kiali
          patches:
          - path: metadata.labels.custom-label
            value:
              deployment-custom-label-value
    tracing:
      enabled: true
      k8s:
        replicaCount: 1
3

Thanks for the hint, it really works! But…it works only on already running components. For example, I’ve added this config:

spec:
  profile: minimal
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
  addonComponents:
    prometheus:
      enabled: true
      k8s:
        replicaCount: 1
        overlays:
        - apiVersion: apps/v1
          kind: Deployment
          name: prometheus
          patches:
          - path: metadata.labels
            value: |
              app.kubernetes.io/name: istio-prometheus
          - path: spec.template.metadata.labels
            value: |
              app.kubernetes.io/name: istio-prometheus

When prometheus (or any other) pod is running, then this config works:

2020-11-05T09:26:13.743739Z	info	installer	Applying Kubernetes overlay: 
- apiVersion: apps/v1
  kind: Deployment
  name: prometheus
  patches:
  - path: metadata.labels
    value: |
      app.kubernetes.io/name: istio-prometheus
  - path: spec.template.metadata.labels
    value: |
      app.kubernetes.io/name: istio-prometheus
2020-11-05T09:26:15.564906Z	info	installer	updating resource: Deployment/istio-system/prometheus
- Processing resources for Addons.
- Processing resources for Addons.
✔ Addons installed

Describe:

$ kubectl describe deployments prometheus -n istio-system
Name:                   prometheus
Namespace:              istio-system
CreationTimestamp:      Thu, 05 Nov 2020 10:21:03 +0100
Labels:                 app=prometheus
                        app.kubernetes.io/name=istio-prometheus
Pod Template:
  Labels:           app=prometheus
                    app.kubernetes.io/name=istio-prometheus

When I remove pods and try to re-install it, it fails:

2020-11-05T09:29:14.529671Z	info	installer	Applying Kubernetes overlay: 
- apiVersion: apps/v1
  kind: Deployment
  name: prometheus
  patches:
  - path: metadata.labels
    value: |
      app.kubernetes.io/name: istio-prometheus
  - path: spec.template.metadata.labels
    value: |
      app.kubernetes.io/name: istio-prometheus
2020-11-05T09:29:16.161452Z	error	installer	failed to create "Deployment/istio-system/prometheus": Deployment.apps "prometheus" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{"app.kubernetes.io/name":"istio-prometheus"}: `selector` does not match template `labels`
- Processing resources for Addons.
✘ Addons encountered an error: failed to create "Deployment/istio-system/prometheus": Deployment.apps "prometheus" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{"app.kubernetes.io/name":"istio-prometheus"}: `selector` does not match template `labels`

Is this kind of behavior expected or is it a bug?

4

it does find and replace, so looks like it fails when it cant find which i think is expected behaviour
I dont know if there is “ignore if not found” flag in overlay

btw … you should let the tool manage deployments eg. istioctl to install/ upgrade
if helm is used then use helm to install/ manage/ remove
if terraform is used then use same tool to manage the deployment