I have an interesting problem concerning integrating Kubernetes Operators into our helm charts.
I have PR 13299 that now installs Kiali via the new Kiali Operator.
It works great… UNTIL! I go to uninstall Istio. Using helm delete causes helm to delete the operator before the operator has a chance to clean up the resources it manages. What this means is the Kiali Operator gets removed but Kiali remains running because the operator has been removed and so it can’t remove Kiali and the helm charts can’t remove Kiali because helm knows nothing about the Kiali resources anymore (because managing those resources has been delegated to the operator!).
I think this might be a general problem with helm delegating the installation of any component thru an operator (I don’t think this is Kiali specific) - helm just doesn’t seem to wait for the operator to “do its thing”.
I see Jaeger tracing has PR 9508 that also introduces its operator in the Istio helm charts - I was curious how they managed to work around this (if they did).
Is there something I might be doing wrong? Or am I understanding the situation correctly?
(to be clear, the problem is when uninstalling Istio. Things works great when you install things via operators - the problems surface when you uninstall Istio).
1 Like
Maybe the tightest integration with specific operators should be to optionally apply their CRs and leave the installation of the operator itself to the user?
The Jaeger PR doesn’t currently deal with this problem - it would have the same issue.
I agree with @loewenstein, the problem would be solved if the Istio helm chart was only managing the CR, rather than the operator+CR. This would also make sense in a soft multi-tenancy scenario, where there could be multiple Istio instances installed, but there should only be a single installation of the operators.
However, getting users to install all of the potential operators that may be required currently, before installing Istio, will be a burden. One option could be using the operatorhub, but that would still require multiple steps for the user. Another would be a separate helm chart that installed all of the operators.
OK, so based on this, it seems the helm framework that Istio uses to install and remove itself is not ready to begin accepting and using operators to install components. I’m thinking I should close that PR and keep the kiali helm chart the way it is.
When an Istio operator is developed/introduced, we can introduce the Kiali operator at that point since it should be easier to integrate it then (one would hope).
We are planning to deprecate the current helm charts, or keep them for demo-only. The istio-ecosystem/istio-installer has a new set of templates to be used with the new istio operator.
Yes, the assumption is that user will choose which components to install - in their own namespaces. The istio operator/installer may install/remove other components - including operators, or may use existing, user managed components.
For example, if user selects prometheus.enabled=true, we would install prometheus operator in an istio-something namespace. If prometheusNamespace=foo is set, istio will assume user managed version. Still working out the details…
1 Like