Single Active Consumer

In certain scenarios, it is critical to consume and process messages published to a queue in the order they arrive. Fortunately, you can easily navigate use cases like that with the Single Active Consumer in LavinMQ.

What is the Single Active Consumer?

A Single Active Consumer is a configuration in LavinMQ that allows only one consumer to receive and process messages from a specific queue at a time. This consumer is what we call the single active consumer.

If/when the active consumer is unavailable, the queue will automatically fail over to the next available consumer registered with the queue.

What benefits does the Single Active Consumer have?

Generally, the Single Active Consumer arrangement has the following benefits:

It guarantees that messages are processed in the exact order they arrive in the queue.
It also guarantees that only one consumer is executing a specific task at any given time.
By allowing only one consumer at a time, it eliminates conflicts that may arise when multiple consumers try to work on the same message simultaneously.

When should I use the Single Active Consumer?

It is advisable to use a Single Active Consumer in scenarios where: - Maintaining the chronological sequence of events is crucial. For example, in a ticketing system, processing ticket requests in the order they were received ensures fair and accurate handling of customer inquiries. - Preventing parallel processing conflicts is required, or when exclusive and sequential execution of tasks is necessary.

Getting started with Single Active Consumer in LavinMQ

Single Active Consumer can be enabled when declaring a queue, with the x-single-active-consumer argument set to true.

The snippet below gives an example with Pika, the Python client:

# Create a connection
params = pika.URLParameters(url)
connection = pika.BlockingConnection(params)
print("[✅] Connection over channel established")

channel = connection.channel() # start a channel
# Declare a single-active-consumer-enabled queue
channel.queue_declare(
  queue="sac",
  arguments={"x-single-active-consumer":True}
)

Typically, when you declare a queue, you’d bind some consumers to it. In the case of a single-active-consumer-enabled queue, the chain of events after registering consumers to the queue would look like this:

The first consumer registered with the queue becomes the single active consumer.
If the active consumer becomes unavailable, the queue will select the next single active consumer from the pool of registered consumers in order of connection.

Exploring the Single Active Consumer feature in LavinMQ

To test the behaviour of the Single Active Consumer described in the preceding section, take the following steps:

Create a basic application that could consume messages from a single-active-consumer-enabled queue, named, sac.
You can re-use our Python sample code for this.
To simulate having multiple consumers bound to the sac queue, run two instances of the consumer in two separate terminals. The output in the two terminals should look like this:

    
    Terminal 1 - Consumer 1

    ->$ python sac_consumer.py
    [✅] Connection over channel established
    [❎] Waiting for messages. To exit press CTRL+C

    
    Terminal 2 - Consumer 2

    ->$ python sac_consumer.py
    [✅] Connection over channel established
    [❎] Waiting for messages. To exit press CTRL+C

Next, create another basic application that could publish four messages to the single-active-consumer-enabled queue, sac.
Again, you can re-use our Python sample code for this. On running the sample code, the output in your console should look like this:

    
    Terminal 3 - Producer

    ->$ python sac_producer.py
    [✅] Connection over channel established
    [📥] Message sent to queue #SAC message 1
    [📥] Message sent to queue #SAC message 2
    [📥] Message sent to queue #SAC message 3
    [📥] Message sent to queue #SAC message 4
    [❎] Connection closed

Just right after the messages have been published, you should see all of them being consumed by the first consumer. The second consumer does not get any messages. This is because the first consumer bound to the sac queue has been made the active consumer.

    
    Terminal 1 - Consumer 1

    ->$ python sac_consumer.py
    [✅] Received #b'SAC message 1'
    [✅] SAC message processed!
    [✅] Received #b'SAC message 2'
    [✅] SAC message processed!
    [✅] Received #b'SAC message 3'
    [✅] SAC message processed!
    [✅] Received #b'SAC message 4'
    [✅] SAC message processed!

    
    Terminal 2 - Consumer 2

    ->$ python sac_consumer.py
    [✅] Connection over channel established
    [❎] Waiting for messages. To exit press CTRL+C

For the last experiment, run the producer again to publish all the messages. This time, terminate the first consumer just right after it receives the second message. This is just to simulate the active consumer crashing.
As soon as you crash the active consumer, the second consumer will become the active consumer and start receiving the remaining messages.

Side note

Good to know information related to Single Active Consumers in LavinMQ.

Consumer Exclusivity vs Single Active Consumer

To begin, the Single Active Consumer is not the only way to ensure that only one consumer could consume and process messages from a specific queue. There is a second mechanism - consumer exclusivity.

With consumer exclusivity, you can register a consumer as the only consumer allowed to consume messages from a specific queue. See the snippet below:

channel.basic_consume(
    "sac",
    callback,
    auto_ack=False,
    exclusive=True
)

If a consumer is already registered as exclusive for a queue, any other consumer trying to register for the same queue will be denied.

This pattern ensures that there is always exactly one consumer processing messages. But here is the catch:

If the exclusive consumer is canceled or dies, it is the responsibility of the application to register a new consumer to continue consuming from the queue.

The Single Active Consumer configuration is similar to consumer exclusivity - you also have only one consumer actively consuming messages from a queue.

However, When using the single active consumer approach, if the active consumer is canceled or dies, the queue will automatically fail over to the next available consumer.

This pattern ensures that there is always exactly one consumer processing messages, but it allows for a smooth transition in case the active consumer becomes unavailable.

Introduction

Configuration

Language Support

AMQP 0-9-1 Overview

More Exchange Types

More Consumer Features

Queue Deep-dive

Other Features

Reliable Message Delivery

High Availability

Monitoring

Management HTTP API

Tutorials

Networking

LavinMQ CLI