Publishers and Consumers

Like most message brokers, LavinMQ has a straightforward architecture. Client applications called the publishers(producers), create(publish/produce) messages and send them to LavinMQ. Other applications called the consumers, connect to LavinMQ to receive(consume) and process the messages. This approach allows the publisher and consumer to be independent of each other.

Let’s explore publishers and consumers a bit further. As you proceed, keep in mind that we will use the terms producers and publishers interchangeably in this guide.

What is a publisher?

In the context of LavinMQ and messaging in general, a publisher refers to a client application or system responsible for creating and sending messages to LavinMQ. Think of it as a sender that generates information or events to be distributed to other systems or applications.

When client applications create and send messages, in the message queueing parlance, we say they publish or produce messages.

What is a consumer?

On the other hand, a consumer is an application or system that connects to LavinMQ and retrieves messages from it for processing. You can consider a consumer as a receiver that accesses and handles the messages published by the publisher. The consumer is designed to receive and process messages asynchronously, which means it can fetch and work on messages at its own pace without needing direct real-time interaction with the publisher.

When client applications receive or retrieve messages, in the message queueing parlance, we say they consume messages.

NOTE: The publisher and consumer could be services that might be running on different computers or technology stacks, processes on the same computer, or even modules of the same application - thus, it is possible to have the same application function as both the publisher and the consumer.

More on publishers

Let’s cover some more specific details around producers.

Publishing messages

To publish messages, clients establish a connection with LavinMQ. This connection serves as a pathway for communication between the client and the message broker. Within this connection, clients create a channel, which is a virtual communication pathway that allows them to interact with the message broker. Refer to our guide on connections and channels for more information.

Once the connection and channel are established, here are the sequence of events that could play out:

Clients can start publishing messages. How this publishing is done, would vary, depending on the client library used. This is what it will look like with Pika:

    
    channel.basic_publish(
      exchange='',
      routing_key=routing_key,
      body=body
    )

When a message is published, it is not directly sent to a specific queue. Instead, it goes through an intermediary component called an exchange.
An exchange is responsible for receiving messages from publishers and routing them to the appropriate queue(s) based on predefined rules called bindings.
The exchange receives the message from the publisher and examines the message’s routing key. The routing key is a value assigned to the message by the publisher, indicating the intended destination or purpose of the message.
Based on the routing key, the exchange determines the appropriate queue(s) to which the message should be routed.

For more information, you can refer to our guide on exchanges and queues

Publishing to a non-existing exchange

If a message is published to a non-existing exchange, the message broker typically discards the message. This happens because the exchange is responsible for routing messages, so if it doesn’t exist, there is no way for the message to reach its intended destination.

The publisher lifecycle

Publishers can be short-lived or long-lived.

When short-lived, publishers establish a connection with the message broker, create a channel within that connection, and publish messages as needed. Once the messages are published, the publisher’s role is considered complete, and it disconnects from the message broker if desired.

While the above arrangement works, it is not ideal. Publishers tend to be long-lived - they open their connection(s) during application startup and live as long as their application runs and by extension their connection.

In essence, publishers tend to establish a persistent connection with the message broker, reuse the same channel for multiple messages, and publish messages in a loop or based on specific events or triggers.

More on consumers

Let’s cover some more specific details around consumers.

Consuming messages

Like prodcuers, to consume messages, clients establish a connection with the LavinMQ. Consumers also create channels within a connection.

Once the connection and channel are established, clients can start consuming messages from specific queues. In order to consume messages there has to be a queue

Consumers access the messages in a queue by subscribing to it within the channel. By subscribing, consumers express their interest in consuming messages from a specific queue. LavinMQ then delivers the messages to the consumer.

When a message is delivered, a handler provided by the user will be called. The type of handler used depends on the client library, which can be either a user-defined function or object following a specific structure.

The snippet below shows a consumer that subscribes to queue named, hello_world and defines a handler named, call_back:

def callback(ch, method, properties, body):
    print(f"[✅] Received #{ body }")

channel.basic_consume(
    "hello_world",
    callback,
    auto_ack=True,
)

When a subscription operation is successful, it returns a subscription identifier (consumer tag). This identifier can be used later to cancel the consumer if needed. But...

What is a consumer tag?

Consumer tags are identifiers used to differentiate multiple consumers. Each consumer can have a unique consumer tag associated with it, allowing the message broker to track and manage the state of individual consumers.

Consumer tags help ensure that each consumer receives the appropriate messages and that the message broker can handle consumer-specific actions, such as acknowledgments or rejections.

Consuming from non-existing queues

If a consumer tries to consume from a queue that doesn’t exist, LavinMQ typically responds with an 404, "NOT_FOUND" error, indicating that the queue is not available. In this case, the consumer needs to ensure that the queue exists before attempting to consume from it.

The consumer Lifecycle

Consumers are designed to be persistent, meaning they receive multiple message deliveries throughout their lifespan. It is not efficient to register a consumer solely for consuming a single message.

Typically, consumers are registered when the application starts up. They tend to remain active as long as their connection is established, and sometimes even throughout the entire lifespan of the application.

More resources on consumers

To learn more about consumers and the different ways you can use them, we encourage you to check out these resources:

Wrap up

We’ve explored the concepts of producers and consumers in LavinMQ and in messaging in general.

Introduction

Configuration

Language Support

AMQP 0-9-1 Overview

More Exchange Types

More Consumer Features

Queue Deep-dive

Other Features

Reliable Message Delivery

High Availability and Backup

Monitoring

Management HTTP API

Tutorials

Networking

LavinMQ CLI