LavinMQ with Celery

Setting up Celery with LavinMQ

Setting up LavinMQ with Celery locally involves a few steps, including installing LavinMQ, configuring Celery, defining tasks, and running the Celery worker. Here’s a step-by-step guide:

Install LavinMQ: First, you need to install LavinMQ on your local machine. You can download and install it from the official website, or you can set it up in the cloud using CloudAMQP.
Install Celery: Open a terminal or command prompt and install Celery using pip:

pip install celery

Create a Celery Configuration: Create a file named celeryconfig.py in your project directory. This file will contain the configuration settings for Celery:

# celeryconfig.py

# Broker settings (LavinMQ)

# Replace BROKER_URL with your LavinMQ server's URL
# Example : 'amqp://guest:guest@localhost:5672//'
# Default password and username for lavinmq user: guest, pass: guest

BROKER_URL = 'lavinmq://<username>:<password>@<lavinmq_host>:<lavinmq_port>/<virtual_host>'  

# Result backend (Optional, if you want to store task results)
# result_backend = 'rpc://

# Recommended settings for local LavinMQ
broker_pool_limit = 1
broker_heartbeat = None
broker_connection_timeout = 30
result_backend = None
event_queue_expires = 60
worker_prefetch_multiplier = 1
worker_concurrency = 4  # Adjust based on your system's capabilities

More Broker Settings

Default: "amqp://"

The default broker URL. This must be a URL in the form of:

transport://userid:password@hostname:port/virtual_host

Only the scheme part ´transport://´ is required; the rest is optional and defaults to the specific transports default values. The transport part is the broker implementation, and the default is amqp, (uses librabbitmq if installed or falls back to pyamqp). The scheme can also be a fully qualified path to your transport implementation:

broker_url = 'proj.transports.MyTransport://localhost'

More than one broker URL of the same transport can also be specified. The broker URLs can be passed in as a single string that’s semicolon delimited:

broker_url = 'transport://userid:password@hostname:port//;transport://userid:password@hostname:port//'

Or as a list:

broker_url = [
    'transport://userid:password@localhost:port//',
    'transport://userid:password@hostname:port//'
]

Create a Celery Instance: In your Python code, create an instance of the Celery application using the configuration from celeryconfig.py:

from celery import Celery

app = Celery('myapp')
app.config_from_object('celeryconfig')

Define Celery Tasks: Define the tasks you want to execute asynchronously using Celery. For example:

@app.task
def add_numbers(x, y):
    return x + y

Launch the Celery Worker: Start a Celery worker to process tasks. Open a terminal, navigate to your project directory, and execute:

celery -A your_module_name worker --loglevel=info

Replace your_module_name with the actual name of the Python module containing the Celery instance and task definitions.

Dispatch Tasks for Execution:

Send tasks for asynchronous execution in your Python code:

from your_module_name import add_numbers

result = add_numbers.delay(5, 3)
print("Task ID:", result.id)

Monitoring with Celery Flower: You can use Celery Flower to monitor and manage your Celery tasks locally. Install Flower using:

pip install flower

Start Flower by running:

celery -A your_module_name flower

Access the Flower web interface by opening your browser and going to http://localhost:5555.

Setting up LavinMQ with Celery locally enables you to process asynchronous tasks efficiently in your Python application. By following these steps, you can harness the capabilities of Celery and a local LavinMQ server for message queuing.

Introduction

Configuration

Language Support

AMQP 0-9-1 Overview

More Exchange Types

More Consumer Features

Queue Deep-dive

Other Features

Reliable Message Delivery

High Availability

Monitoring

Management HTTP API

Tutorials

Networking

LavinMQ CLI

LavinMQ with Celery

Setting up Celery with LavinMQ

More Broker Settings

Managed LavinMQ instance on CloudAMQP

Help and feedback