Dead Letter Exchange

There are times when a message becomes undeliverable and is dropped by the broker, for instance, when the amount of time the message has spent in a queue exceeds the time to live ( TTL) when the queue reaches its capacity, or when a message is negatively acknowledged by the consumer. Such a message is called a dead message. The dead letter exchange enables a way to take care of these dead messages instead of letting them be completely dropped by the broker.

What is the Dead Letter Exchange?

The Dead Letter Exchange is a way to handle messages that, for some reason, are marked as dead messages. The dead message can end up in a queue and stay there until it is consumed by another consumer.

What benefits does the Dead Letter Exchange have?

Queues attached to a dead letter exchange collect dropped messages with the next steps determined by you as to the desired outcome.

When should I use the Dead Letter Exchange?

Attach a dead letter exchange when it is known that there will be messages that are nacked but still need to be handled. A dead letter exchange should also be set up when you can’t lose messages with an expiring TTL or when the queue might reach its capacity.

The Dead Letter Exchange

When is a message considered dead?

There are three identified situations where a message becomes undeliverable after reaching LavinMQ:

A message is negatively acknowledged by the consumer
The TTL of a message expires
The queue reaches capacity

By default, the broker drops these messages. Publishing is successful, however, the LavinMQ consumer never handles or has a chance to handle the message successfully.

Queues attached to a dead letter exchange collect dropped messages for processing according to instructions you set. In other words, it’s up to you to decide how to handle messages in the dead letter queue. When implemented correctly, information is almost never lost.

Attach a dead letter exchange when it is known that there will be messages that might be nacked but still need to be handled. A dead letter exchange should also be set up when you can’t lose messages with an expiring TTL or when the queue might reach its capacity.

Setting up a LavinMQ dead letter exchange

Dead letter exchanges are no different than other exchanges. Simply specify the exchange normally and declare it as a backup for a queue:

channel.exchange_declare("dlx_exchange", "direct")
channel.queue_declare("test_queue", arguments={
  "x-dead-letter-exchange": "dlx_exchange", "x-dead-letter-routing-key": "dlx_key"})

The x-dead-letter-exchange parameter tells the test_queue to use the dlx_exchange for dead messages. Notice how the exchange is not dedicated to a single queue.

Creating and Binding LavinMQ Dead Letter Queues

Just as with a dead letter exchange, a dead letter queue is a regular queue in LavinMQ, it is just attached to the exchange.

Create the queue normally and attach it to the exchange:

channel.queue_declare("dead_letter_queue")

If desired, specify a new routing key:

channel.queue_bind("dead_letter_queue", "dlx_exchange", "dlx_key")

The x-dead-letter-routing-key changes the routing key from the one used in the original message to dlx_key. This allows you to redirect dead letter messages from multiple queues to the same dead letter queue.

Retrying dead letters

You can create a redelivery mechanism using dead letter queues by attaching the original exchange to the dead letter queue. Be aware, however, that it is possible to create an endless cycle of redelivered messages which could clog dead letter queues if left unchecked.

Create a separate dead letter queue that stores messages before pushing them to an exchange or routing them back to the original queue.

To further improve the system include an incremented property in the message body indicating the number of times the message was received. This requires the system to handle dead letters in a separate consumer but allows messages to eventually drop or be pushed into storage.

Mixing with a delayed exchange

To further improve dead letter handling, an LavinMQ plugin exists to specify a delay in an exchange. A delayed message exchange introduces the x-delayed-message type passed at creation time:

channel.exchange_declare("dlx_exchange", "direct")
channel.queue_declare("test_queue", arguments={
    "x-dead-letter-exchange": "dlx_exchange", "x-dead-letter-routing-key": "dlx_key"})

Specify the x-delay header in messages, using successive delay times to avoid overburdening LavinMQ. This lets you simplify the process of dead letter handling.

Catching orphaned messages in LavinMQ

Whether caused by a negative acknowledgment, expiration of the time to live, or exceeded queue length, losing information is detrimental for many applications. Knowing when and how to use the LavinMQ dead letter exchange avoids losing undelivered messages.

Introduction

Configuration

Language Support

AMQP 0-9-1 Overview

More Exchange Types

More Consumer Features

Queue Deep-dive

Other Features

Reliable Message Delivery

High Availability and Backup

Monitoring

Management HTTP API

Tutorials

Networking

LavinMQ CLI