The AMQP 0.9.1 Protocol

Why another messaging protocol?

AMQP emerged from my experience and frustrations with integrating front- and back-office systems at investment banks — the same problems of connecting systems together would crop up with depressing regularity. From 1996 through to 2003 I was waiting for the solution to this obvious requirement to materialize as a standard, and thereby become a commodity. But that failed to happen, and I grew impatient. Consequently, AMQP was created.

That was John O’Hara, the first creator of AMQP, describing its motivation in his article, Toward a Commodity Enterprise Middleware.

Since its creation, AMQP has evolved — from v0.8 through v0.9.1 and, more recently, v1.0. Since older protocol versions are obsolete, this blog will focus on v0.9.1.

AMQP 0.9.1

AMQP 0.9.1 is a wire protocol, a widely used open standard for message-oriented middleware. It defines rules that, when implemented correctly, enable reliable and secure communication between systems, allowing them to send and receive messages. AMQP 0-9-1 is considered a compact protocol since it’s a binary protocol, meaning that everything sent over AMQP is binary data.

But how does AMQP 0-9-1 work?

AMQP 0-9-1 follows a client-server architecture — you have client libraries that implement the client side of the protocol and a broker ( like LavinMQ ) that implements the server side of the protocol. Applications leverage these client libraries to send or receive data to or from the broker.

In the AMQP space, client applications that send data to the broker are called producers. Client applications that receive data from the broker are called consumers. The data sent or received to/from the broker are called messages. The broker acts as an middleman between these producers and consumers.

One of AMQP 0.9.1’s core objectives is to allow messaging interoperability between systems, regardless of the message broker vendor or platform used. So, for example, you should be able to switch conveniently from RabbitMQ to LavinMQ, without updating your clients very much, since both brokers implement the AMQP 0.9.1 protocol.

💡 Moving on, it is important that AMQP 0.9.1 should not just be considered a protocol only used for over-the-wire communication; it defines both:

• The Server Model, formally called the AMQ model: a high-level architecture for message brokers.
• And the Network Protocol, formally called AMQP: The rules of engagement between clients and the server - defining how clients talk to the broker and the AMQ model it implements.

We will organize the remainder of this blog into these two key areas.

AMQP 0-9-1 server model

For the purpose of interoperability, AMQP defines the core components that handle message routing and storage within the broker, along with the rules for connecting them. It is strictly required that these components be the same in any given server implementation. This consistency allows one AMQP 0-9-1 broker to be largely interchangeable with another.

The specification outlines four key components of an AMQP-based broker:

• A Virtual host
• An Exchange
• Bindings & Routing keys
• And a Queue

A virtual host is a separate data section within the broker. It includes its own unique name, a collection of exchanges, message queues, and all related objects.

Messages from producers are first published to a broker component called an exchange. Essentially, when a producer sends a message, it specifies the exchange to which it will be published and provides a routing key for the message as well.

The exchange routes the messages it receives to a second component of the broker called the queue. - the exchange has to do this for every message received. Exchanges are fundamentally message routing agents. What queue an exchange decides to send messages to usually depends on the type of the exchange - different exchanges have different message routing strategies.

However, in general, all exchanges use a combination of queue bindings and a message’s routing key to decide what queue to send the message to. Let’s explore the concepts of bindings and routing keys.

Bindings

Bindings are connections established between exchanges and queues. They define the relationship between the two, specifying which queues should receive messages from a particular exchange. In other words, bindings specify what queues are interested in the messages coming to a given exchange.

When you create a queue, you can bind it to one or more exchanges using a binding key. The binding key acts as a filter or criteria that determines whether a message should be routed to a particular queue or not. AMQP 0.9.1 has clearly defined protocol commands for these:

# Create an exchange called "test_exchange"
Exchange.Declare
	exchange=test_exchange

# Create a queue called "test_queue"
Queue.Declare
	queue=test_queue


# Bind the queue to the exchange: queue_bind(name, exchange, binding_key)
Queue.Bind
	queue=test_queue
	exchange=test_exchange
	routing-key=test

Routing keys

Routing keys are properties assigned to messages by producers. They are essential for routing messages from exchanges to queues. A routing key is a string value that carries information about the message’s intended destination.

With a better understanding of these concepts, we can now picture the flow of messages from a producer to a consumer as shown in the image below.

💡 As a recap, when a message arrives at an exchange, the exchange would examine its routing key. It then compares the routing key with the binding keys of all queues bound to that exchange. Once a match is found, the broker delivers the message to the associated queue. However, what is considered a match between a message’s routing key and a queue’s binding key entirely depends on the exchange type.

Let’s examine the different exchange types outlined in the AMQP 0-9-1 protocol specification.

Exchange types

Exchanges in AMQP 0.9.1 are like traffic controllers for your messages, ensuring they reach the right queue or queues. When a producer sends a message, it first hits an exchange, which then decides which queue the message should end up in. There are four main types of exchanges, each with its own style:

• Direct exchanges are like precise GPS systems, routing messages to queues with exact matching routing keys.
• Fanout exchanges are like broadcasters, sending messages to all queues bound to the exchange without even looking at the routing keys.
• Topic exchanges are like matchmakers, routing messages based on wildcard patterns in the routing key.
• Finally, Headers exchanges are the custom routers, using message headers to make routing decisions.

Each type of exchange offers a unique and powerful way to control how messages flow through your AMQP 0.9.1 setup. A guide explaining exchanges can be found in our blog Exchanges, touring keys and bindings.

We’ve repeatedly mentioned how exchanges route messages to queues, but..

What are Queues really?

A queue in the context of AMQP, is a named FIFO buffer that stores messages for consumer applications. These applications can create, share, use, and delete message queues within their permissions.

Message queues can be durable, temporary, or auto-deleted. Durable message queues remain until they are deleted. Temporary message queues exist until the server shuts down. Auto-deleted message queues are removed when they are no longer in use.

Message queues store their messages in memory, on disk, or both. Each virtual host has its own named message queues.

Message queues store messages and distribute them to one or more consumer clients. A message sent to a queue is delivered to only one client unless it needs to be resent due to a failure or rejection. A single message queue can hold different types of content simultaneously.

The network protocol: AMQP

The AMQP 0.9.1 specification also defines the rules of engagement between a client and an AMQP broker. These rules allow client applications to talk to the broker and interact with entities like vhosts, exchanges and queues that the broker defines.

The protocol is organized into two distinct layers: the functional layer and the transport layer.

• The functional layer: provides commands grouped into logical classes, enabling useful operations like managing transactions, exchanges, and message queues. This layer defines what the protocol can do for applications. Below this layer lies the transport layer.
• The transport layer: ensures that these commands are delivered seamlessly between applications and servers. It handles tasks like channel multiplexing, framing data, encoding content, and managing errors. Additionally, it supports features like heartbeats to maintain connection health and represents data in a consistent format.

An interesting aspect of AMQP’s design is its flexibility. The transport layer can be swapped out for other transport mechanisms without affecting the application’s functionality. Similarly, this transport layer could support other protocols, demonstrating its adaptability.

As mentioned earlier, the network protocol generally allows client applications to talk to a broker, thereby performing tasks like:

• The creation and management of connections and channels
• The construction, publishing and management of messages in general
• Consuming messages
• Handling errors

Connections and Channels

The AMQP 0.9.1 protocol defines some set of protocol frames(commands) that allow clients to establish a connection to an AMQP broker and then channels over the established connection. Connection and Channels?

Well, to send/receive messages to/from the broker, producers and consumers open a connection to the message broker via the client library it integrates. In AMQP, a connection is like a virtual link between your application (the producer or consumer) and the message broker, opening a communication line between the two.

An application wants to interact frequently with the message broker; connections can be expensive to open and close. So, instead of always opening and closing connections, you can reuse a connection and open something called a Channel.

Channels allow you to have many smaller connections on one physical connection. Think of channels as lightweight pathways within the connection, each capable of carrying its own stream of messages. By using channels, you can reuse a connection and avoid opening and closing connections between the apps and the message broker.

For connections and channels, the AMQP 0.9.1 defines protocol commands for client → broker interaction. The list below is not exhaustive - it only serves to give you an idea of what these commands look like:

Connection.Start, Connection.Tune, Connection.Open, Channel.Open, Channel.Close

Every connection must be linked to a virtual host.

Messages

A message is an entity sent from the publisher to the queue and finally consumed by the consumer.

• Headers: Messages have a set header defining properties like: content-type, and delivery-mode . Messages could also include custom headers for additional metadata.
• Body Content: The message body contains the actual payload, which can be any binary format.

Clients use the Basic.Publish protocol command to send a message to an exchange. This frame includes the routing key, exchange, and other relevant information.

Basic.Publish
    exchange=my_exchange
    routing-key=my.routing.key

Wrap up

Systems often leverage a messaging protocol like AMQP for inter-service communication. While there are two up-to-date variants of the AMQP protocol, this guide covers the AMQP 0-9-1. This variant of the protocol defines rules that allow producers and consumers talk to a broker over channels created on a connection.

Beyond what happens outside a broker, the protocol also defines the key components that an AMQP-based broker must implement: exchanges, queues, bindings and virtual hosts.

Ready to get started with an AMQP broker?

Read the LavinMQ beginner tutorials to get familiar with LavinMQ. Visit our Slack channels for community updates, follow our social media channels, and subscribe to our newsletter for the latest news.