Offline configuration validation

Continuing the discussion from Configuration validation:

I think something Istio is missing is static validation of configuration objects. It should be possible to validate at least the syntax of resource definitions in a CI toolchain and without access to a live service mesh.

This is a problem with Kubernetes as well, as described here and addressed here.

I know there’s no quick fix right now, but I suspect a lot of people have lost a lot of time to debugging indent levels in YAML files and if there was a path to generate OpenAPI or JSON Schema documents for Istio CRDs, that class of problem could be solved with already existing software.

FYI, we did add istioctl validate -x that can perform referential integrity check for Mixer-related CRDs. It performs deep structural and referential validation:

  • expression parsing and type checking within conditions
  • references to attributes, instances, and handlers are checked in rules

FYI, Kiali as a goal to help Istio configuration providing additional validation.

Specially those semantic aspects that requires multiple checking with other Istio or k8s resources (i.e. if I define a Gateway, does that Gateway exist?).

https://www.kiali.io/documentation/validations/

The idea is to grow on these features with the help of community feedback, so any ideas are welcome.

I’ve been working on a validation model that work both offline, as well as through K8s API Server.
Here is the description: https://docs.google.com/document/d/1m19jkLyiIXNmUH_TYClRH-IOmuCJO4-vduy5KaCa99E/edit#heading=h.qex63c29z2to

There is a proof of concept here, but it has rotted a bit:

With this model, we can run the same set of validation rules against resources in both the command-line, as well as when they are ingested.
Separately, we’re working on stream-lining per-resource validation, which will utilize API Server’s CRD validation schema. It should be possible to use the same model in the command-line as well, in a consistent manner.

There is an ongoing document about the validation approach that Jason Wang is driving:
https://docs.google.com/document/d/1ugwCNgQa4oxZK1JZ8NiaV7PAh4rI7H5MNwoF6LgOZIE/edit#heading=h.9t5cvudb0ru7

That looks like something I should be able to use right now in my CI pipeline. It provides some istio-related advice on standard k8s resource definitions too, which is great.

I’m assuming I’ll need to pull the istioctl binary out of the istio release tarball for whichever
istio release I’m currently running? I don’t see an obvious docker image to run these tests with.

Kiali looks great for online monitoring, and I need to use it more. Thanks for the prompt on that.

I can’t see the Google docs, but that sounds very useful and I look forward to trying it out.

Thanks everyone for all the pointers and advice.