The Art of Auctioning: A Competitive Edge in Integration Patterns

Welcome back, integration enthusiasts! After delving into the depths of the Scatter-Gather pattern with its distribution style, it’s time to move on to the next captivating integration pattern. As promised, today we will explore the Auction variant of the Scatter-Gather pattern from Gregor Hohpe and Bobby Woolf’s influential work, Enterprise Integration Patterns..

Just like before, we’ll continue using MuleSoft as our primary implementation tool, and we’ll also introduce RabbitMQ into the mix to demonstrate how to create a competitive environment where services bid for their place. Ready for this thrilling ride? Let’s get started!

The Scatter-Gather

“How do you maintain the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply?”

Use a Scatter-Gather that broadcasts a message to multiple recipients and re-aggregates the responses back into a single message.

Auction Style Scatter-Gather: A Quick Recap

Before we jump into the implementation, let’s take a moment to understand what makes the Auction style different from the Distribution style we discussed earlier. The Distribution style is all about broadcasting messages to multiple recipients and gathering all responses without discrimination. Every response matters equally, and we combine them to form a single output.

But what if not all responses are created equal? What if we want to pick the best response out of a set, perhaps the fastest, cheapest, or most efficient? This is where the Auction style comes into play.

What is Auction Style Scatter-Gather?

In the Auction style, we send a message to multiple recipients who then "bid" by sending back their responses. Rather than combining all responses into one, we evaluate these bids and select the best one according to predefined criteria, such as cost, performance, or availability.

Key Characteristics of Auction Style

• Selective Processing: Unlike Distribution style, Auction style doesn’t necessarily aggregate all responses; it selects the most suitable one based on certain criteria.
• Competitive Environment: Recipients compete by sending their best offers, similar to an auction.
• Efficiency: By picking the optimal response, the Auction style helps optimise resource utilisation and service quality.
• Dynamic Decision Making: This style allows dynamic decision-making based on real-time data and responses from different services.

Our Use-Case

Let’s expand on the previous logistical use-case. Suppose now, instead of just finding the cheapest carrier service, we want to ensure the best balance between cost and delivery speed. We’ll have our carriers “bid” for the job, and we’ll select the most cost-effective and speedy service.

In implementing these patterns with MuleSoft, the approach differs depending on whether you are using the Distribution or Auction style of the Scatter-Gather pattern.

For the Distribution style, MuleSoft provides a built-in Scatter-Gather component that broadcasts a message to multiple recipients and aggregates all responses within a single flow. This is ideal for scenarios where every response is equally important and needs to be combined into a cohesive output.

In contrast, the Auction style uses a more nuanced setup. Instead of the Scatter-Gather component, it leverages a publish-subscribe model combined with MuleSoft’s message aggregators. This method broadcasts messages via a Publish-Subscribe Channel, allowing only interested participants (those ready and willing to respond) to receive the request. The message aggregators then collect and evaluate these responses to select the best bid based on criteria like cost or delivery time, creating a dynamic and flexible bidding environment.

MuleSoft's Scatter-Gather Auction in Action!

Introducing RabbitMQ into the Auction Process

To effectively implement an Auction style pattern, we need a reliable message broker to handle the communication between our system and the multiple recipients. RabbitMQ is a perfect fit for this role due to its robust messaging capabilities and support for various messaging patterns.

In this scenario, RabbitMQ will act as an intermediary, receiving the initial request from our system, broadcasting it to the potential service providers (bidders), and receiving their responses (bids) and forwarding them to the reply queue.

Let's see how MuleSoft orchestrates all of this.

Trigger the flow

For this scenario we use an http endpoint to trigger the Mule flow. The flow sets a payload to send to the carriers. In reality, this dummy payload would contain information about the parcel and its destination.

Please reply to...

The message includes a replyTo channel so that the carriers know where to send there bids. When using RabbitMQ we can use the replyTo channel to transparently send the response to this queue from the carriers.

Place your bids!

Two flows have been setup subcriber1Flow (pictured below) and subscriber2Flow. These represent the carriers handling the bids and returning the quote to our system. Normally, these flows would not be part of the Mule application but are added as part of the application for convenience.

Receive Bids

The Mule application receives all the bids through the messaging reply channel. Each response is processed by a group-based aggregator, which organises the responses by their correlation IDs. The aggregation continues until all responses in a group are collected, either reaching the desired group size or when a timeout occurs.

Winning bidder

After the aggregation is complete, the Mule aggregation listener is activated for the specific group, and the bid with the lowest cost is selected from the collected responses.

Wrapping Up: Auction Style in Action

The Auction style Scatter-Gather pattern provides a more sophisticated approach to message processing by allowing us to dynamically select the best response based on real-time data. By incorporating the concept of “interested participants” through a publish-subscribe model, we optimise our system’s performance and ensure that only relevant responses are considered, which is especially valuable in scenarios with numerous potential bidders.

Stay tuned as we continue to explore more enterprise integration patterns and their practical applications. In future posts, we’ll cover additional patterns and techniques to help you enhance your integration solutions.

As always, the MuleSoft source code and RabbitMQ configuration accompanying this blog will be made available here. Until next time, happy integrating and keep bidding and winning in the world of enterprise integration!