Ability to change Istio app/version labels?


#1

Istio recommends use of ‘app’ and ‘version’ labels for the purpose of tracing.
https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/

Kubernetes recommends use of app.kubernetes.io/name and app.kubernetes.io/version https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/

The problem for me lies in using ELK as labels get translated into fields and app and app.kubernetes.io/name collide with each other. Is there any config somewhere I can modify such that I can change the app and version labels istio reads to the kube recommended labels?

Thanks


#2

There was/is a PR to support those labels - but it was deferred due to the large churn it was going to introduce in the code base during this 1.1 release cycle.

I can tell you Kiali can be told which app/version labels you want to use when visualizing things (default is still app/version, but you can configure it to whatever names you want) but you still need Istio to use those custom labels too.

Today, you CAN do what you want, but you have to change the Istio metrics CR so it looks for your custom labels and not the defaults.


#3

Example places of where you’d need to customize the labels (this isn’t the only place, just one of them):


#4

Fantastic info. TYVM


#5

Made necessary changes to the kiali and istio configs and everything is working great.

Thanks!


#6

@jmazzitelli should we resurrect that PR (on the master branch)?


#7

@douglas-reid for selfish reasons, I would say no because it will make my life harder :smiley: But seriously - I would say no because supporting multiple labels at the same time is going to cause problems for any client of the telemetry (not just for Kiali, but anyone trying to analyze the telemetry) - it will require multiple k8s requests to determine what pods were participating in a particular timeseries within the telemetry (because some could be labeled with app, some with app.kubernetes.io/name, some with the /instance label - then we get into the different version labels).

An alternative solution would be to have the helm charts accept optional arguments so you can define what app and version labels you will be using in your mesh. So I can say at install time, "I am not using “app” or “version” - instead I will use “app.kubernetes.io/name” and “app.kubernetes.io/version”:

helm --set telemetery.app.label.name=app.kubernetes.io/version --set telemetry.version.label.name=app.kubernetes.io/version

The helm charts that build the mixer metric definitions would use these to define the attribute expressions.

This would have the added benefit it providing a way for users to define their OWN custom labels - because you know someone is going to say, “My company uses the label “application” (or fill in the blank for whatever label name someone is going to want to use) for the app label - how can I tell Istio to use that?”


#8

I hesitate to reject emerging standards / recommendations simply because it means more work.

I not certain why switching labels has any impact on determining pod involved in requests. We have the workload information in the metrics already. Presumably all the pods in a workload have the same labels for app and version, right? Is the concern that there will be pods without a deployment controller labelled using different labels schemes serving the same service?

Adding more config options is an interesting idea – one worth pursuing. I still think we should transition to defaulting to the recommended labels moving forward, however.


#9

Is the concern that there will be pods without a deployment controller labelled using different labels schemes serving the same service?

Yes. See my comment and gargnupur’s followup here: https://github.com/istio/istio/pull/10807#issuecomment-458659223

Specifically, how would one answer the question “what pods are considered to be a part of app ‘foo’”? To support multiple labels would require multiple, separate k8s queries since selectors do not support doing something like “app=foo OR app.kubernetes.io/name=foo OR app.kubernetes.io/instance=foo” (today that would require 3 separate queries). Same thing for multiple version labels as well.

I still think we should transition to defaulting to the recommended labels moving forward, however.

Completely agree. I think eventually switching over to the defaults of the standard names – app.kubernetes.io/name,version – is something we should eventually get to.


#10

Keep in mind Istio supports multiple platforms and raw VMs.

Can we just translate the labels in pilot, so k8s workloads use k8s style label but gets translated to istio style?


#11

I believe we started down that path before, and then pulled back. I don’t recall why at this point but in general I don’t think we care too much about how the attribute destination.labels is ultimately populated – just that it is populated. If we end up creating some sort of metadata service (maybe split off from pilot), that’d be equally good I imagine.