Introduction
Configuration
Language Support
AMQP 0-9-1 Overview
More Exchange Types
More Consumer Features
Queue Deep-dive
Other Features
Reliable Message Delivery
High Availability and Backup
Monitoring
Management HTTP API
Tutorials
Networking
LavinMQ CLI
Publishers and Consumers
Like most message brokers, LavinMQ has a straightforward architecture. Client applications called the publishers(producers), create(publish/produce) messages and send them to LavinMQ. Other applications called the consumers, connect to LavinMQ to receive(consume) and process the messages. This approach allows the publisher and consumer to be independent of each other.
Let’s explore publishers and consumers a bit further. As you proceed, keep in mind that we will use the terms producers and publishers interchangeably in this guide.
What is a publisher?
In the context of LavinMQ and messaging in general, a publisher refers to a client application or system responsible for creating and sending messages to LavinMQ. Think of it as a sender that generates information or events to be distributed to other systems or applications.
When client applications create and send messages, in the message queueing parlance, we say they publish or produce messages.
What is a consumer?
On the other hand, a consumer is an application or system that connects to LavinMQ and retrieves messages from it for processing. You can consider a consumer as a receiver that accesses and handles the messages published by the publisher. The consumer is designed to receive and process messages asynchronously, which means it can fetch and work on messages at its own pace without needing direct real-time interaction with the publisher.
When client applications receive or retrieve messages, in the message queueing parlance, we say they consume messages.
NOTE: The publisher and consumer could be services that might be running on different computers or technology stacks, processes on the same computer, or even modules of the same application - thus, it is possible to have the same application function as both the publisher and the consumer.
More on publishers
Let’s cover some more specific details around producers.
Publishing messages
To publish messages, clients establish a connection with LavinMQ. This connection serves as a pathway for communication between the client and the message broker. Within this connection, clients create a channel, which is a virtual communication pathway that allows them to interact with the message broker. Refer to our guide on connections and channels for more information.
Once the connection and channel are established, here are the sequence of events that could play out:
- Clients can start publishing messages. How this publishing is done, would vary, depending on the client library used. This is what it will look like with Pika:
- When a message is published, it is not directly sent to a specific queue. Instead, it goes through an intermediary component called an exchange.
- An exchange is responsible for receiving messages from publishers and routing them to the appropriate queue(s) based on predefined rules called bindings.
- The exchange receives the message from the publisher and examines the message’s routing key. The routing key is a value assigned to the message by the publisher, indicating the intended destination or purpose of the message.
- Based on the routing key, the exchange determines the appropriate queue(s) to which the message should be routed.
For more information, you can refer to our guide on exchanges and queues
Publishing to a non-existing exchange
If a message is published to a non-existing exchange, the message broker typically discards the message. This happens because the exchange is responsible for routing messages, so if it doesn’t exist, there is no way for the message to reach its intended destination.
The publisher lifecycle
Publishers can be short-lived or long-lived.
When short-lived, publishers establish a connection with the message broker, create a channel within that connection, and publish messages as needed. Once the messages are published, the publisher’s role is considered complete, and it disconnects from the message broker if desired.
While the above arrangement works, it is not ideal. Publishers tend to be long-lived - they open their connection(s) during application startup and live as long as their application runs and by extension their connection.
In essence, publishers tend to establish a persistent connection with the message broker, reuse the same channel for multiple messages, and publish messages in a loop or based on specific events or triggers.
More on consumers
Let’s cover some more specific details around consumers.
Consuming messages
Like prodcuers, to consume messages, clients establish a connection with the LavinMQ. Consumers also create channels within a connection.
Once the connection and channel are established, clients can start consuming messages from specific queues. In order to consume messages there has to be a queue
Consumers access the messages in a queue by subscribing to it within the channel. By subscribing, consumers express their interest in consuming messages from a specific queue. LavinMQ then delivers the messages to the consumer.
When a message is delivered, a handler provided by the user will be called. The type of handler used depends on the client library, which can be either a user-defined function or object following a specific structure.
The snippet below shows a consumer that subscribes to queue named, hello_world
and defines a handler named, call_back
:
When a subscription operation is successful, it returns a subscription identifier (consumer tag). This identifier can be used later to cancel the consumer if needed. But...
What is a consumer tag?
Consumer tags are identifiers used to differentiate multiple consumers. Each consumer can have a unique consumer tag associated with it, allowing the message broker to track and manage the state of individual consumers.
Consumer tags help ensure that each consumer receives the appropriate messages and that the message broker can handle consumer-specific actions, such as acknowledgments or rejections.
Consuming from non-existing queues
If a consumer tries to consume from a queue that doesn’t exist, LavinMQ typically
responds with an 404, "NOT_FOUND"
error, indicating that the queue is not available.
In this case, the consumer needs to ensure that the queue exists
before attempting to consume from it.
The consumer Lifecycle
Consumers are designed to be persistent, meaning they receive multiple message deliveries throughout their lifespan. It is not efficient to register a consumer solely for consuming a single message.
Typically, consumers are registered when the application starts up. They tend to remain active as long as their connection is established, and sometimes even throughout the entire lifespan of the application.
More resources on consumers
To learn more about consumers and the different ways you can use them, we encourage you to check out these resources:
Wrap up
We’ve explored the concepts of producers and consumers in LavinMQ and in messaging in general.
Ready to take the next steps? Here are some things you should keep in mind:
Managed LavinMQ instance on CloudAMQP
LavinMQ has been built with performance and ease of use in mind - we've benchmarked a throughput of about 1,000,000 messages/sec. You can try LavinMQ without any installation hassle by creating a free instance on CloudAMQP. Signing up is a breeze.
Help and feedback
We welcome your feedback and are eager to address any questions you may have about this piece or using LavinMQ. Join our Slack channel to connect with us directly. You can also find LavinMQ on GitHub.