Production multi-cluster installation

Hi, I’ve been working on an Istio multi-cluster implementation that could be as minimal as possible and at the same time open for future challenges/features.

I’m running on AWS and I’m moving to a VPC flat network implementation using aws cni plugin.
For this iteration no multi cloud, just multi-cluster in same or via peered VPC with no CIDR overlap.

Currently I’m not interested in features like mTLs, telemetry or policies… just pure networking.

In my case a service can run in a single or in multiple clusters (if HA is requested) and what I’d like to achieve in this first iteration is:

  • offer GRPC load-balancing among services that run in multiple clusters leveraging the flat networking model
  • keep the cluster traffic local when a consumer/serviceHA live in the same cluster.
  • drain traffic from a service deployed in HA from a cluster to another.

I’d like the help of the community about following questions I have:

  • Start with a minimal Istio installation or full?
  • How difficult and production ready is to add other components like citadel, galley, mixer, egresses… later?
  • Start with a single control plane or multi?
  • Is there a way to keep the flat network between clusters going for a multi control plane configuration? I’ve seen only docs with egress/ingresses configurations.
  • What do you guys think about ? It seems to me that I could save time in glueing conf among clusters… but the project is in alpha and the adoption is low for what I can gather from the repository.

So far I’ve experimented with and I reached a minimal install that has only pilot, ingress-gateway, and the sidecar-injector… i’m not able yet to keep the traffic local… my guess is because pilot thinks it is a single big cluster… would love to get some inputs on this as well…

Hope this post will be helpful for the whole community, since I was able to find so little about production ready multi-cluster installations/experiences.