Orchestrating microservices

In today’s dynamic software landscape, microservices architecture has emerged as a powerful approach to building scalable and resilient applications. However, breaking down a monolithic application into smaller, independent services introduces new challenges, particularly in managing the interactions and dependencies between these services. Orchestrating microservices effectively becomes crucial for ensuring a seamless and efficient flow of data and functionality across the entire system. This involves coordinating the execution of multiple services to achieve a specific business outcome. Without proper orchestration, you risk creating a distributed monolith, negating the benefits of microservices like independent deployment and technology diversity. We’ll explore how robust orchestration strategies can unlock the full potential of your microservices architecture and help you achieve agility and scalability.

Understanding Microservices Orchestration

Microservices orchestration is the process of coordinating the interactions between multiple microservices to complete a larger task or transaction. It involves defining the sequence and dependencies of service calls, handling failures, and managing state across services. This differs from choreography, where each service knows its responsibilities and reacts to events without a central coordinator. Orchestration typically relies on a central component, often called an orchestrator, to manage the workflow. The orchestrator defines the execution order, manages retries, and handles error scenarios. This centralized approach offers better visibility and control over the overall process.

A key benefit of orchestration is its ability to handle complex business processes that require multiple services to work together. For instance, an e-commerce order processing system might involve services for inventory management, payment processing, shipping, and customer notifications. Orchestration ensures that these services are invoked in the correct order, with proper error handling and compensation mechanisms in place. According to a recent report by Gartner, “By 2025, more than 60% of enterprises will utilize microservices-based architectures for new and modernized applications, up from 30% in 2021.” This statistic highlights the growing importance of effective microservices orchestration strategies. Choosing the right orchestration pattern significantly impacts the overall performance and reliability of your application.

Consider a scenario where a user places an order. The orchestrator first calls the inventory service to check stock availability. If the stock is sufficient, it calls the payment service to process the payment. Upon successful payment, it triggers the shipping service to initiate the shipment process and then notifies the customer through the notification service. If any step fails, the orchestrator can trigger compensation transactions to revert any changes made by previous services, ensuring data consistency. This is critical for maintaining data integrity and a positive user experience. The orchestrator acts as the conductor, ensuring each service plays its role harmoniously.

Benefits of Using Microservices Orchestration

Implementing a well-designed microservices orchestration strategy offers numerous advantages. Firstly, it simplifies complex workflows by providing a centralized point of control and visibility. This makes it easier to understand and manage the interactions between services. Secondly, orchestration improves fault tolerance by allowing for retries, compensation transactions, and circuit breakers. If a service fails, the orchestrator can attempt to recover or gracefully degrade the system. This is where concepts like “saga pattern”, a sequence of local transactions, come into play. Each transaction updates data within a single service, and if one transaction fails, the saga executes compensating transactions to undo the previous changes. This ensures data consistency across distributed services.

Furthermore, orchestrating microservices enhances scalability by enabling independent scaling of individual services. The orchestrator can monitor the performance of each service and dynamically adjust the number of instances based on demand. This ensures that resources are allocated efficiently and that the system can handle peak loads. Moreover, orchestration promotes loose coupling between services, making it easier to evolve and update individual services without impacting the entire system. This aligns with the core principles of microservices architecture, allowing for faster development cycles and increased agility. Orchestration tools can include everything from basic messaging queues to sophisticated workflow engines. Choosing the right tool set is critical for success.

Here are some key benefits summarized:

• Simplified Workflow Management: Centralized control and visibility.
• Improved Fault Tolerance: Retries, compensation, and circuit breakers.
• Enhanced Scalability: Independent scaling of individual services.
• Loose Coupling: Easier evolution and updates of individual services.

Common Orchestration Patterns and Tools

Several orchestration patterns and tools are available, each with its own strengths and weaknesses. One common pattern is the Saga pattern, which we touched on earlier, used for managing long-running transactions across multiple services. There are two main types of Sagas: choreography-based and orchestration-based. In a choreography-based saga, each service listens for events and performs its local transaction accordingly. In an orchestration-based saga, a central orchestrator manages the workflow and instructs each service on what to do. Another popular pattern is the Command Query Responsibility Segregation (CQRS), which separates read and write operations to optimize performance and scalability. CQRS is often used in conjunction with Event Sourcing, where all changes to an application’s state are stored as a sequence of events.

Several tools can be used for microservices orchestration, including Apache Kafka, Kubernetes, and workflow engines like Camunda and Activiti. Apache Kafka is a distributed streaming platform that can be used to build event-driven architectures and orchestrate services through event streams. Kubernetes is a container orchestration platform that can manage the deployment, scaling, and networking of microservices. Workflow engines provide a graphical interface for designing and executing complex workflows. The choice of tool depends on the specific requirements of the application, including the complexity of the workflows, the desired level of scalability, and the available resources. According to a survey by the Cloud Native Computing Foundation (CNCF), Kubernetes is the most widely used container orchestration platform, with over 83% of respondents using it in production. CNCF provides valuable resources to understand cloud native technologies.

Here’s an example of a simple orchestration workflow using a workflow engine:

1. The orchestrator receives a request to process an order.
2. The orchestrator calls the inventory service to check stock.
3. If stock is available, the orchestrator calls the payment service.
4. If payment is successful, the orchestrator calls the shipping service.
5. The orchestrator notifies the customer and updates the order status.

Best Practices for Successful Orchestration

To ensure successful microservices orchestration, it’s essential to follow some best practices. First, clearly define the boundaries and responsibilities of each service. This helps to minimize dependencies and ensures that each service has a single, well-defined purpose. Second, design for failure. Implement retries, circuit breakers, and compensation transactions to handle failures gracefully and prevent cascading failures. Third, monitor and log all service interactions. This provides valuable insights into the performance and behavior of the system and helps to identify and resolve issues quickly. Centralized logging and monitoring are crucial for maintaining visibility into the distributed system. Tools like Prometheus and Grafana can be used for monitoring, while tools like ELK stack (Elasticsearch, Logstash, Kibana) can be used for centralized logging.

Fourth, use asynchronous communication where possible. Asynchronous communication allows services to communicate without blocking each other, improving performance and scalability. Message queues like RabbitMQ or Apache Kafka can be used for asynchronous communication. Fifth, automate the deployment and management of microservices. This reduces the risk of human error and ensures that the system can be deployed and updated quickly and reliably. Tools like Jenkins and GitLab CI/CD can be used for automating the deployment process. Sixth, consider security implications at every stage of the process, and don’t expose internal microservices directly to the external world. Instead, use an API Gateway. For example, Kong or Tyk. See the OWASP guide to API Security for more information.

Finally, avoid creating a “distributed monolith”. This happens when services are too tightly coupled and cannot be deployed or scaled independently. By carefully designing the service boundaries and using asynchronous communication, you can avoid this pitfall. Remember that the goal of microservices architecture is to achieve agility and scalability, and effective orchestration is key to realizing these benefits. One key takeaway is that microservices should be independently deployable.

FAQ About Microservices Orchestration -------------------------------------

What is the difference between orchestration and choreography?

Orchestration involves a central coordinator that manages the interactions between services, while choreography relies on services reacting to events without a central coordinator. Orchestration provides better control and visibility, while choreography offers more flexibility and autonomy.

What are some common orchestration patterns?

Common patterns include the Saga pattern, CQRS, and Event Sourcing. The Saga pattern is used for managing long-running transactions, CQRS separates read and write operations, and Event Sourcing stores all changes as a sequence of events.

What tools can be used for microservices orchestration?

Tools include Apache Kafka, Kubernetes, and workflow engines like Camunda and Activiti. The choice of tool depends on the specific requirements of the application.

The journey of adopting microservices and implementing effective orchestration requires careful planning and execution. By understanding the principles, patterns, and tools involved, you can build scalable, resilient, and agile applications that meet the demands of today's dynamic business environment. Implementing these strategies will ultimately lead to more robust and efficient systems. Remember, a well-orchestrated microservices architecture is not just about technology; it's about aligning technology with business goals to achieve tangible results.

Ready to take your microservices architecture to the next level? Explore the resources mentioned above and consider experimenting with different orchestration patterns and tools to find what works best for your specific needs. Consider reading more about event-driven architecture, or perhaps delve into the intricacies of API design. Your next step towards building a truly scalable and resilient system starts now. For real-world examples, refer to NGINX’s introduction to microservices.

Question & Answer :
What is the standard pattern of orchestrating microservices?

If a microservice only knows about its own domain, but there is a flow of data that requires that multiple services interact in some manner, what’s the way to go about it?

Let’s say we have something like this:

• Invoicing
• Shipment

And for the sake of the argument, let’s say that once an order has been shipped, the invoice should be created.

Somewhere, someone presses a button in a GUI, “I’m done, let’s do this!” In a classic monolith service architecture, I’d say that there is either an ESB handling this, or the Shipment service has knowledge of the invoice service and just calls that.

But what is the way people deal with this in this brave new world of microservices?

I do get that this could be considered highly opinion-based. but there is a concrete side to it, as microservices are not supposed to do the above. So there has to be a “what should it by definition do instead”, which is not opinion-based.

Shoot.

The Book Building Microservices describes in detail the styles mentioned by @RogerAlsing in his answer.

On page 43 under Orchestration vs Choreography the book says:

As we start to model more and more complex logic, we have to deal with the problem of managing business processes that stretch across the boundary of individual services. And with microservices, we’ll hit this limit sooner than usual. […] When it comes to actually implementing this flow, there are two styles of architecture we could follow. With orchestration, we rely on a central brain to guide and drive the process, much like the conductor in an orchestra. With choreography, we inform each part of the system of its job and let it work out the details, like dancers all find‐ ing their way and reacting to others around them in a ballet.

The book then proceeds to explain the two styles. The orchestration style corresponds more to the SOA idea of orchestration/task services, whereas the choreography style corresponds to the dumb pipes and smart endpoints mentioned in Martin Fowler’s article.

Orchestration Style

Under this style, the book above mentions:

Let’s think about what an orchestration solution would look like for this flow. Here, probably the simplest thing to do would be to have our customer service act as the central brain. On creation, it talks to the loyalty points bank, email service, and postal service […], through a series of request/response calls. The customer service itself can then track where a customer is in this process. It can check to see if the customer’s account has been set up, or the email sent, or the post delivered. We get to take the flowchart […] and model it directly into code. We could even use tooling that implements this for us, perhaps using an appropriate rules engine. Commercial tools exist for this very purpose in the form of business process modeling software. Assuming we use synchronous request/response, we could even know if each stage has worked […] The downside to this orchestration approach is that the customer service can become too much of a central governing authority. It can become the hub in the middle of a web and a central point where logic starts to live. I have seen this approach result in a small number of smart “god” services telling anemic CRUD-based services what to do.

Note: I suppose that when the author mentions tooling he’s referring to something like BPM (e.g. Activity, Apache ODE, Camunda). As a matter of fact, the Workflow Patterns Website has an awesome set of patterns to do this kind of orchestration and it also offers evaluation details of different vendor tools that help to implement it this way. I don’t think the author implies one is required to use one of these tools to implement this style of integration though, other lightweight orchestration frameworks could be used e.g. Spring Integration, Apache Camel or Mule ESB

However, other books I’ve read on the topic of Microservices and in general the majority of articles I’ve found in the web seem to disfavor this approach of orchestration and instead suggest using the next one.

Choreography Style

Under choreography style the author says:

With a choreographed approach, we could instead just have the customer service emit an event in an asynchronous manner, saying Customer created. The email service, postal service, and loyalty points bank then just subscribe to these events and react accordingly […] This approach is significantly more decoupled. If some other service needed to reach to the creation of a customer, it just needs to subscribe to the events and do its job when needed. The downside is that the explicit view of the business process we see in [the workflow] is now only implicitly reflected in our system […] This means additional work is needed to ensure that you can monitor and track that the right things have happened. For example, would you know if the loyalty points bank had a bug and for some reason didn’t set up the correct account? One approach I like for dealing with this is to build a monitoring system that explicitly matches the view of the business process in [the workflow], but then tracks what each of the services do as independent entities, letting you see odd exceptions mapped onto the more explicit process flow. The [flowchart] […] isn’t the driving force, but just one lens through which we can see how the system is behaving. In general, I have found that systems that tend more toward the choreographed approach are more loosely coupled, and are more flexible and amenable to change. You do need to do extra work to monitor and track the processes across system boundaries, however. I have found most heavily orchestrated implementations to be extremely brittle, with a higher cost of change. With that in mind, I strongly prefer aiming for a choreographed system, where each service is smart enough to understand its role in the whole dance.

Note: To this day I’m still not sure if choreography is just another name for event-driven architecture (EDA), but if EDA is just one way to do it, what are the other ways? (Also see What do you mean by “Event-Driven”? and The Meanings of Event-Driven Architecture). Also, it seems that things like CQRS and EventSourcing resonate a lot with this architectural style, right?

Now, after this comes the fun. The Microservices book does not assume microservices are going to be implemented with REST. As a matter of fact in the next section in the book, they proceed to consider RPC and SOA-based solutions and finally REST. An important point here is that Microservices does not imply REST.

So, What About HATEOAS? (Hypermedia as the Engine of Application State)

Now, if we want to follow the RESTful approach we cannot ignore HATEOAS or Roy Fielding will be very much pleased to say in his blog that our solution is not truly REST. See his blog post on REST API Must be Hypertext Driven:

I am getting frustrated by the number of people calling any HTTP-based interface a REST API. What needs to be done to make the REST architectural style clear on the notion that hypertext is a constraint? In other words, if the engine of application state (and hence the API) is not being driven by hypertext, then it cannot be RESTful and cannot be a REST API. Period. Is there some broken manual somewhere that needs to be fixed?

So, as you can see, Fielding thinks that without HATEOAS you are not truly building RESTful applications. For Fielding, HATEOAS is the way to go when it comes to orchestrating services. I am just learning all this, but to me, HATEOAS does not clearly define who or what is the driving force behind actually following the links. In a UI that could be the user, but in computer-to-computer interactions, I suppose that needs to be done by a higher level service.

According to HATEOAS, the only link the API consumer truly needs to know is the one that initiates the communication with the server (e.g. POST /order). From this point on, REST is going to conduct the flow, because, in the response of this endpoint, the resource returned will contain the links to the next possible states. The API consumer then decides what link to follow and move the application to the next state.

Despite how cool that sounds, the client still needs to know if the link must be POSTed, PUTed, GETed, PATCHed, etc. And the client still needs to decide what payload to pass. The client still needs to be aware of what to do if that fails (retry, compensate, cancel, etc.).

I am fairly new to all this, but for me, from HATEOAs perspective, this client, or API consumer is a high order service. If we think it from the perspective of a human, you can imagine an end-user on a web page, deciding what links to follow, but still, the programmer of the web page had to decide what method to use to invoke the links, and what payload to pass. So, to my point, in a computer-to-computer interaction, the computer takes the role of the end-user. Once more this is what we call an orchestrations service.

I suppose we can use HATEOAS with either orchestration or choreography.

The API Gateway Pattern

Another interesting pattern is suggested by Chris Richardson who also proposed what he called an API Gateway Pattern.

In a monolithic architecture, clients of the application, such as web browsers and native applications, make HTTP requests via a load balancer to one of N identical instances of the application. But in a microservice architecture, the monolith has been replaced by a collection of services. Consequently, a key question we need to answer is what do the clients interact with?

An application client, such as a native mobile application, could make RESTful HTTP requests to the individual services […] On the surface this might seem attractive. However, there is likely to be a significant mismatch in granularity between the APIs of the individual services and data required by the clients. For example, displaying one web page could potentially require calls to large numbers of services. Amazon.com, for example, describes how some pages require calls to 100+ services. Making that many requests, even over a high-speed internet connection, let alone a lower-bandwidth, higher-latency mobile network, would be very inefficient and result in a poor user experience.

A much better approach is for clients to make a small number of requests per-page, perhaps as few as one, over the Internet to a front-end server known as an API gateway.

The API gateway sits between the application’s clients and the microservices. It provides APIs that are tailored to the client. The API gateway provides a coarse-grained API to mobile clients and a finer-grained API to desktop clients that use a high-performance network. In this example, the desktop clients make multiple requests to retrieve information about a product, whereas a mobile client makes a single request.

The API gateway handles incoming requests by making requests to some number of microservices over the high-performance LAN. Netflix, for example, describes how each request fans out to on average six backend services. In this example, fine-grained requests from a desktop client are simply proxied to the corresponding service, whereas each coarse-grained request from a mobile client is handled by aggregating the results of calling multiple services.

Not only does the API gateway optimize communication between clients and the application, but it also encapsulates the details of the microservices. This enables the microservices to evolve without impacting the clients. For example, two microservices might be merged. Another microservice might be partitioned into two or more services. Only the API gateway needs to be updated to reflect these changes. The clients are unaffected.

Now that we have looked at how the API gateway mediates between the application and its clients, let’s now look at how to implement communication between microservices.

This sounds pretty similar to the orchestration style mentioned above, just with a slightly different intent, in this case, it seems to be all about performance and simplification of interactions.