Saga Pattern Demonstration: Order System in AWS

The Saga pattern is crucial in microservices architectures for managing long-running transactions and ensuring data consistency across distributed services. It breaks a complex business process into smaller, isolated steps, each of which has its own transaction and compensating actions in case of failure. This pattern avoids the need for monolithic transactions or distributed locks, and instead relies on smaller, independently managed steps that can recover from failures while maintaining overall system consistency.

In this demo that I've written (my GitHub), the Saga pattern is demonstrated through an order processing system built using AWS services, such as Step Functions, Lambda, ECS Fargate, and PostgreSQL. The architecture follows the Saga pattern, where each service is responsible for one step in the order process and each has its own datastore. Here’s a breakdown of the architecture and how it works:

System Architecture

Architecture Overview:

1. AWS Step Functions: Orchestrates the entire workflow, coordinating each service and managing the transitions between states based on the success or failure of each operation. It handles the sequence of steps for placing an order, checking inventory, processing payment, and taking compensatory actions when necessary.

2. API Gateway & Lambda: Users interact with the system via an API Gateway. Each API request triggers a Lambda function, which acts as a wrapper around the corresponding Spring Boot microservice running in ECS Fargate. These Lambda functions act as intermediaries that invoke the respective microservices to handle specific tasks, like order creation, inventory check, and payment processing.

3. ECS Fargate: The microservices (Order Service, Inventory Service, and Payment Service) are containerized and deployed in Amazon ECS Fargate. Each service runs in its own container and interacts with its respective PostgreSQL database for data persistence. Fargate handles the container orchestration, ensuring that the microservices run independently without managing the underlying infrastructure.

4. Datastores (PostgreSQL): Each microservice has its own PostgreSQL database:

   • Order Service stores the order details.

   • Inventory Service manages product stock levels.

   • Payment Service keeps track of payment status.

5. State Machine:

   • Order Service: When the user places an order, this Lambda function creates an order in the Order Service and moves the process to the next state (Check Inventory).

   • Inventory Service: This service checks the inventory to determine if enough stock is available. If stock is available, it triggers the Payment Service. If stock is insufficient, the process triggers the Cancel Order state.

   • Check Inventory: The Check Inventory step evaluates whether the inventory is sufficient. If it is, the workflow proceeds to the Payment Service. If not, Cancel Order is invoked.

   • Payment Service: This service attempts to process the payment. If successful (status 200), the order moves forward. If the payment fails (status 402), the Release Inventory step is invoked to restore inventory levels.

   • Release Inventory: If payment fails, this compensatory step restores the inventory that was previously deducted by the Inventory Service.

   • Cancel Order: If inventory is insufficient or payment fails, the order is canceled, marking the transaction as failed.

The State Machine orchestrates the flow of events, ensuring that compensating actions are executed as needed. For instance, if payment fails or inventory is unavailable, the process automatically calls the Release Inventory or Cancel Order steps to ensure the system remains in a consistent state.

State transitions for PaymentService failing with 402 (Payment Required)

How the Saga Pattern Works in This Architecture

1. User Interaction: A user places an order through the REST endpoint on the API gateway, which triggers the Lambda function for the Order Service.

2. Step Functions Workflow:

   • The Order Service Lambda invokes the Order Service running on ECS Fargate to create an order.

   • The Inventory Service Lambda checks if enough inventory is available. If inventory is sufficient, it proceeds to Payment Service.

   • The Payment Service Lambda processes payment and returns either a success or failure. If payment fails, it triggers Release Inventory to restore stock levels. If payment is successful, the saga completes.

3. Compensation: If at any step an error occurs (e.g., insufficient inventory or payment failure), the system triggers compensatory actions such as releasing inventory or canceling the order, ensuring consistency across services.

4. Data Persistence: Each service operates on its own PostgreSQL database, ensuring that data is decoupled between services. Inventory changes and payment statuses are stored and managed independently, reducing the risk of data conflicts and ensuring fault tolerance.

By using AWS Step Functions to orchestrate this process, the system can handle long-running workflows and ensure that each step is properly executed or compensated if it fails. The integration with ECS Fargate ensures that the services are managed in containers, while the use of Lambda functions as wrappers allows for seamless integration of microservices into the overall Saga workflow.

This project showcases the Saga pattern in action, demonstrating how to build a resilient, scalable, and maintainable order processing system in a microservices environment. It highlights the benefits of using AWS services to orchestrate complex workflows and manage distributed transactions across multiple services.

To download the project and run it in your own AWS environment, visit my GitHub here: https://github.com/mjones3/order-system