Let’s say we have four microservices, each represented by four states, we can have a whole family of microservices for each state, so we have high availability at the microservice layer. Each microservice will not have its own database, I want each microservice to attack the same data cluster following the CQRS pattern.
In the layer above those microservices, I have a layer that acts as an orchestrator, that manages the transactional state of each request, each request will obviously have a unique identifier.
Now, when the cycle starts, I want the orchestrator to invoke only one of the type A family micro-services, when that type A family can potentially have as many type A family docker containers as we see necessary, but I want only one to be selected to handle the request.
I want to be able to invoke the microservice asynchronously, I don’t want to use anything like synchronous REST, so I understand that I must use a subscriber publishing system, for example kafka.
To give an example, I am talking as broadly as possible without being tied to any technology. I understand that each transactional state is going to be tied to a topic, let’s say for “state A”, it would be called topicA and if we have several microservices to attend the transactional “state A”, at the end we would have as many topics as states and microservices of a family, that is, topicA-1, topicA-2,…, topicA-n, topicB-a, topicB-2,…,topicB-n,…topicD-1,topicD-2,…topicD-n.
So I understand that in the end I need a load balancing between the orchestrator and the publisher subscriber layer to select the topic I’m going to send the message to.
Now I think that Istio could do the job, but I am not sure…
Is there a load balancer at the subscriber publisher level? Ok, I know that this is more a Kafka related question, sorry about that.
Is Istio able to do the load balancing between partitioned subtopics or unpartitioned topics.
Is there an api to programmatically interact with Istio daemon?
Ideally, partitions can be created for each topic, so that one topic manages a transactional state corresponding to each microservice. Each partition within the topic would correspond to a microservice, so far so good, now, how could you do it dynamically? because potentially speaking there will be times when it is detected that only n microservices of one type are needed, at other times they may be more, less and that will require dynamically managing in the load balancer those sub-partitions in each topic, so that it can create them, delete them, assign that partition to a microservice, tell the controller class that it can use that partition to communicate asynchronously with that microservice, manage the health of that microservice and act accordingly?
I ask all this because I’m a rookie, this morning I started reading and playing with the project
Thank you very much.