A short guide to message queueing
Written by: Lovisa Johansson
Message queueing is a form of asynchronous communication between services or applications constructed of messages that are held in a queue until they are processed.
Message queues are used to decouple processes for system efficiency, coding simplicity, and improved reliability. This is the ultimate guide to message queueing including what a message queue is, how to use one, and the amazing benefits achieved by using a message queue.
What is message queueing?
A queue is a series of things waiting to be processed in sequential order. A message queue, therefore, is a series of messages waiting to be processed. These messages consist of data sent between applications. The message often includes a set of instructions or tasks that should be performed by the receiver of the message.
A message queue’s construction is simple. On one side there are client applications known as producers with the task of creating messages and delivering them to the message queue. On the other, a consumer application connects to the queue to retrieve messages for processing. Messages on the queue remain there until they are sent or processed depending on how they are acknowledged by the consumer.
How do message queues work?
Message queues provide asynchronous communications protocol, meaning that a system places a message on the queue and does not require a response to continue its tasks. A perfect example of asynchronous communication is emailing. After sending an email, the sender does not wait for a response to continue on with other tasks. By handling messages in this fashion, the producer and the consumer are decoupled and do not need to interact with the message queue at any certain time.
How is decoupling an advantage?
Maintenance and delayed processing
In a decoupled system like a message queue, functions are kept separate and self-contained, they do not rely on each other in order to get their individual job done. This is achieved when two or more systems can communicate without being connected to each other. Because the system’s components are kept separate, this makes the system easier to maintain. If one part of the system fails to process messages from the queue due to routine maintenance or downtime, other parts of the system can continue to add messages and do their own processing and the system can catch up when it has recovered while experiencing no data loss.
A decoupled system is also advantageous when setting up delayed processing. For instance, a producer delivers a message to a queue, which is not consumed until an appointed time which is controlled by the broker.
Decoupled systems and scalability
Preferred over a large coupled application, decoupled systems can communicate asynchronously using e.g. message queues, allowing for different parts of the system to be upgraded independently, communicate through different languages, or be developed by separate teams. The point of message queues is to keep the individual processes operating independently of each other. One process will never need another process in order to do its assigned task, it just places the message on the queue or consumes it from the queue, and continues on to the next task.
If messages are being delivered to the queue at a faster pace than the consumers can handle them, the queue will just continue to grow. Under heavy load, if the queue grows larger, one can simply add more consumers and parallelize the work. This is a simple and effective method of scaling.
How to use message queues, an example
As an example, imagine a web service receiving multiple requests per second, like a online pizza service. It cannot afford to lose any messages and must also be highly available and ready for new requests instead of locked up by previous messages. In this use case, placing a message queue between the web service and a processing service creates the ideal environment.
The web service places messages (pizza orders) on the queue and other processes take it from there, handling messages in order and keeping the original service free to accept new messages (new orders). The two processes are decoupled, so they can handle multiple requests in a short time even if the number of messages gets large (like on Friday evenings). As the business increases its volume and needs to scale up the system, the development team just needs to add more consumers to work off the messages in the queue.
Go with LavinMQ
If you’re considering a message queue-based solution, and want to get up an run quickly, look to CloudAMQP to host your LavinMQ server. LavinMQ is an open-source message-oriented middleware solution implementing AMQP (advanced message queueing protocol), with features like queueing, routing, reliability, and security.