I would like to take some time to clarify Istio’s plans for Helm support in 1.5, 1.6, and beyond. By way of background, Istio 1.3 introduced the Istio Operator as an experimental feature which allows administrators greater ease of use and flexibility when installing and operating Istio. The Operator is accessed through the istioctl manifest command, and will enter Beta in the 1.5 release. Under the hood, this operator is built on top of Helm charts, but these charts are completely distinct from the earlier charts, which were packaged in the binary at install/kubernetes/helm/istio.
In Istio 1.6, the original helm charts will be removed from the project. The new charts are a more modular version of the original charts, which relied on subcharts. Moving forward we’re adding support for multiple control plane revisions and modular installation, which are not possible with the old all-in-one charts. The recommended path for installing Istio and managing Istio installations will be using the Operator via Istioctl. For users who would like to continue installing and managing Istio via Helm directly, we will support calling the Helm charts which underlie the operator, and are located at istio/istio/manifests. In 1.5 the charts require Helm2 - with either ‘template’ or tiller. In 1.6 we’ll support both helm3 and helm2. This will increase parity between Helm and non-Helm use cases, and reduce drift between the various installation and management paths.
I hope this information helps to clarify our plans for Helm support. Feel free to reply with any related questions!
Hope that with 1.6, all the options that helm chart installation method provides will be available with istioctl method,
eg.
we couldn’t install ingress object for kiali with istioctl and had to install it separately, in our use-case kiali is made available via nginx ingress, helm method supports that.
another one was installing user (custom) ingress g/w isn’t supported with istioctl currently.
Note that in the near future it is likely that third-party add-on components like Kiali will not be possible via istioctl installer, other than perhaps through a “demo” profile of some kind. The alternative would be to go outside istioctl and use the third-party installers (e.g. in Kiali’s case, using the Kiali Operator to perform the installation of Kiali).
Why, when im looking inside of install/kubernetes/helm/istio/values.yaml i still see galley,mixer,pilot still enabled , no references to the istiod component and mtls auto not enabled. I thought in 1.5 they are disabled and mtls is on? @Mitch_Connors
What is the best course of action to get to the new way of managing Istio if coming from a Helm installed setup? Do i remove Istio and reinstall, create new cluster altogether, run Operator (installed with Helm charts) and then start using Istioctl?
I like that we are seeing what the helm support looks like, but I’m more interested in how to move forward in a safe manner.
Yeah, this is my concern too. I’ve actually given this a go today and it’s really not gone well at all. I did it in a test cluster, but it had loads of down time, and I still couldn’t get it working. After installing Istio via Istio Operator I tried to remove the Helm release and it took some of the stuff the Istio Operator had created with it (and then the Istio Operator didn’t try recreate those things either, I had to delete and then recreate my IstioOperator resource for it to install everything again). When I did get Istio up and running again, it didn’t seem to work with the values that had worked with Helm. The HPA for the ingress gateway was pointing at the wrong ref, my ingress gateway was 404ing, and I had some errors around certificates not being present any more (despite restarting all pods), around cert manager CRDs not being present (I’ve got a separate installation of Cert Manager using the latest version?), and not being able to make a PVC for Grafana. I never figured out the 404 issue despite trying for hours, so I went back to using Helm.
I only hope that if the Helm Chart really has been removed that there are some really clear instructions on how to proceed with this migration. Something must have been wrong with my setup, obviously, but if so, there needs to be some clear indication of what somewhere.
I have tested on a dev cluster the upgrade path from istio 1.4 installed with helm to 1.5 using the IstioOperator CR. I had the same issues when removing the helm chart. It deleted some resources created by the operator, and I was surprised that the operator didn’t try to recreate them. Moreover, I think that the operator ignores some options define in the CR, although they are documented. I will retest this and fill and issue.
Hi, there’s not a non-disruptive migration path from 1.4 helm charts to 1.5 istioctl or operator because the new helm charts that are used under the hood by these are not compatible with the old charts. If you just take the new charts in 1.5 (manifest/.) and try to apply them on top of the old charts (install/kubernetes/.), it will not work as there are too many changes. This is really nothing to do with istioctl/operator but rather the refactoring of the underlying helm charts into a new format, which is not compatible with in-place upgrade from the old charts. You cannot upgrade to the new charts even using helm.
1.4 users have to either stick with using helm install for 1.5 (using the old charts) or completely reinstall with istioctl or operator. For 1.6, it will be possible to migrate smoothly because there will be the ability to have multiple control plane versions run side by side and roll over the data plane from one to the other. @Aaron-ML, the operator can be installed in 1.5 using either istioctl operator init or using the operator helm charts in the release tar. We’ve removed the kubectl -f option because there were too many install options for operator and this one was of limited value, since all the values were hard coded.
Yes, this doc is accurate. In 1.5, you can install with legacy helm charts, which will install galley, pilot, etc, or you can install using the new charts or operator, which will install istiod. Upgrading your install from microservices (galley, pilot, etc) to istiod is not fully supported until 1.6.
Okay, just wasn’t sure if it was expected to deprecate 1.4 install methods moving forward since it’s in the 1.4 docs.
Also, is there any helm repo for the istio operator that we can pull? It’s going to be very cumbersome to manually cherrypick that chart just to get to the latest operator.
Thank you for this detailed explanation, @Mitch_Connors.
We are hoping to migrate to the Istio operator in 1.6 - will the istio-operator Helm chart be published to the Istio Helm chart repository? That would be valuable for users that want to continue using Helm as a declarative deployment tool.
Hi, the istio-operator chart can be found in the release tar under install/kubernetes/operator/charts/istio-operator, and for 1.6 onwards it’s under manifests/charts/istio-operator.
1.4 users have to either stick with using helm install for 1.5 (using the old charts) or completely reinstall with istioctl or operator. For 1.6, it will be possible to migrate smoothly because there will be the ability to have multiple control plane versions run side by side and roll over the data plane from one to the other.
I don’t think that’s correct, the operator does support multiple control planes in 1.6. Maybe @sdake is thinking of additional features we had planned in the operator, like enhancements for rolling over between different revisions, or else I may be missing something. istioctl and operator have always used the same templates and now use largely the same code both to generate the manifests and actuate the changes in the cluster, so I’d be surprised if their behavior was different.
However, when I do install via istioctl using the istioctl manifest apply method, there looks to be an “installed-state” IstioOperator in istio-system. So is it actually using the IstioOperator? This is just a little confusing to me because I installed Istio using the non-operator method so I wondered if I’m misunderstanding something.
Also, it looks like there are a number of ways to install with istioctl: istioctl manifest apply, istioctl manifest generate then apply with kubectl, istioctl profile dump then apply with kubectl, istioctl install, and Helm. From previous Istio releases, I’m most familiar with the Helm install method; however, I’d like to use whatever the go-forward recommendation is.
What’s the recommended installation approach where I have a more configuration granularity than simply setting a profile, and allow me to set tune specific parameters in the form of a config file (without saving off gigantic yaml files from istioctl generate). Basically what is the modern equivalent of the the Helm values.yaml file? I saw that it looks like you could define an IstioOperator object to give you a good amount of configurability, but I wasn’t sure if I should be using that given the aforementioned warning in the docs about using the IstioOperator in production.
I’d be very grateful for any clarification you have to offer regarding this.