We are designing our platform to be compatible with different cloud providers (GCloud, OCI, Azure etc). Our platform is built using micro-services and we have our 30 of them. We recently began migration to Kubernetes. Our problem is the following, some of our services are to be run in both our’s and customer’s infrastructure. For example, if we have three services X, Y & Z in our platform. If X is deployed on customer infrastructure and Y and Z on our infrastructure, how do I best set up networking amongst X, Y & Z? Currently we have had some success with HashiCorp’s Consul
using it as a key-value store. Basically X will query Consul
for Y’s endpoint. I am not sure if this is the best possible solution. I wonder if this problem is solvable through other tools such as KubeDNS
or a distributed key-value store more cleanly: maybe run etcd
node on customer’s infrastructure and our’s? Another complication that might arise is that Y and Z will not be in the same Kubernetes namespace on our infrastructure. For instance Y might be a web server serving static files. Where as Z might be a service we have deployed on behalf of the customer in our infrastructure and all three X, Y and Z must be able to talk to each other. Currently we are using Istio
for service mesh (and TLS offloading). Is there a solution that leverages Istio
in this case?