Python hello world

This part is also available in: Node.js Ruby

You might be familiar with what LavinMQ is already. But to reiterate, we are going to go over it again, but this time around, in an approachable manner.

What is LavinMQ

LavinMQ is a message broker also called a message queue - But what is a message broker?

From a non-technical standpoint, a broker is literally a middle man that facilitates a transaction between two parties, usually a buyer and a seller. Message brokers do pretty much the same thing in the world of software.

LavinMQ and other brokers like it, facilitate the transfer of binary blobs of data, known as messages, between a sending application and a receiving application interested in the message. But how?

The sending application, known as a producer, creates and sends messages to LavinMQ. This process is referred to as publishing.
As a message broker, LavinMQ receives and securely stores messages from producers until they are retrieved by receiving applications. LavinMQ utilizes its built-in queue that maintains the order of arrival and persists messages until they are picked up.
- To better get a sense of how a qeueue works in LavinMQ and in fact in most message brokers, you can think of it as a cylinder with open ends. New messages are added to the queue from one end of the cylinder and only removed from the opposite end - First In First Out(FIFO).
A receiving application formally called the consumer receives messages from LavinMQ and processes them at its own pace. This process is referred to as consuming.

The image below visualizes the flow of messages from the Producer -> LavinMQ -> Consumer.

LavinMQ First In First Out Queue

Some noteworthy information on producers, consumers and messages: - Producers and consumers can be processes running on the same computer, modules of an application, or services running on different computers with varying technology stacks. - Messages are the data added by the sender application to the queue. They can include task requests for the receiving system, like sending an email or resizing an image, as well as plain text or information about completed tasks.

Now that we get the basics of LavinMQ, let’s see how we can set up a basic producer that publishes messages to LavinMQ and a corresponding consumer that consumes the messages.

Publishing and consuming messages with Pika, Python client

We will build the simplest thing you can make with LavinMQ - two rudimentary Python applications. The first Python application, the producer will publish messages to LavinMQ and the second application, the consumer will receive these messages from LavinMQ.

There are three steps we will take to build our rudimentary producer and consumer: 1. Setting up the Python development environment - we will only have this step in this section. For subsequent Python tutorials, we will reference this guide and have you replicate steps where necessary. 1. Creating the producer 1. Creating the consumer

Python development environment

First make sure that you have Python and that it’s available from your command line. You can confirm this by running: python --version
If you get a NameError: name 'python' is not defined, then you do not have Python. Download Python from python.org or you could even use pyenv to manage multiple Python versions
Run python3 -m venv env-name on Unix and macOS or python -m venv env-name on Windows to create a virtual environment where we install the dependencies for our project. Replace env-name with whatever name you chose for your virtual environment.
Run source env-name/bin/activate on Unix and macOS or .\env-name\Scripts\activate on Windows to activate the virtual environment.
Create a directory for your LavinMQ tutotirls Run mkdir lavinmq_tutorials && cd lavinmq_tutorials to create a directory named lavinmq_tutorials and then navigate into it - this directory will house the tutorial in this section and all the subsequent ones. We will create sub-directories in this folder to organize the different tutorials based on technologies.
next create a sub-directory for the python tutorials in the lavinmq_tutorials directory. Run mkdir python_tutorials && cd python_tutorials to create a sub directory named python_tutorials and then navigate into it
Open your project directory in your favourite text-editor and create a requirements.txt file in the python_tutorials sub-directory. Add pika==1.1.0 and python-dotenv to requirements.txt file
Install dependencies with pip install -r requirements.txt
Create a .env file in in the root directory
Add CLOUDAMQP_URL="lavinmq_url" to the `.env’ file. Replace lavinmq_url with your correct server url

Creating the producer - sending messages

Here, we will create a basic python application that publishes a plain text message to LavinMQ - the hello world of LavinMQ.

There are four steps we need to create an application that sends a message to LavinMQ: - First, we need to create a connection to our LavinMQ instance from our Python code - Next, we create a channel within the connection we’ve created - Next, we declare a queue - Then publish messages to the queue

Let’s see what these steps look like in code

Creating a connection

Create a file hello_world_producer.py in the python_tutorials sub-directory. and add the snippet below to the file:

import pika, os
from dotenv import load_dotenv

load_dotenv()

# Access the CLOUDAMQP_URL environment variable and parse it (fallback to localhost)
url = os.environ.get('CLOUDAMQP_URL', 'amqp://guest:guest@localhost:5672/%2f')

In the snippet above, we imported relevant packages and loaded our Lavinmq connection string from the .env file. Next, let’s create a connection to our Lavinmq server. Add the code snippet below to the end of hello_world_producer.py file.

# Create a connection
params = pika.URLParameters(url)
connection = pika.BlockingConnection(params)

Create a channel

Add the code snippet below to the end of hello_world_producer.py file, to create a channel within the connection we’ve created in the previous step.

channel = connection.channel()
print("[✅] Channel over a connection created")

Now that we have a connection and a channel, let’s try to make sense of what they are.

If we think of sending a message from a producer to a Lavinmq instance in terms of, let’s say, delivering a package between two houses in a neighbourhood, then we can say a connection is the direct road linking these two houses and a channel, the multiple paths within that road.

In more technical terms, a connection is the direct link between a producer or consumer and a Lavinmq server, giving the producer or consumer access to the Lavinmq server.

Because you are most likely going to be needing to send or receive multiple messages, a channel allows the re-use of a connection - think of channels as smaller connections within a connection - all messages are published and consumed on a channel.

Declare a queue

Add the code snippet below to the end of hello_world_producer.py file, to declare a queue, named hello_world

channel.queue_declare(queue='hello_world')

hello_world in the snippet above is the queue name. In the event that you do not pass a queue name, one would be auto-generated.

Publish messages to the declared queue

We are going to create a re-usable function that we can use to publish messages to LavinMQ. Add the code snippet below to the end of hello_world_producer.py file, to create a function for publishing messages, publish three messages, “Hello World” and close the connection.

def send_to_queue(channel, routing_key, body):
  channel.basic_publish(
      exchange='',
      routing_key=routing_key,
      body=body
  )
  print(f"[📥] Message sent to queue - msg:  #{body}")

# Publish messages
send_to_queue(
    channel=channel, routing_key="hello_world", body="Hello World"
)
send_to_queue(
    channel=channel, routing_key="hello_world", body="Hello World"
)
send_to_queue(
    channel=channel, routing_key="wrong_routing_key", body="Hello World"
)
try:
  connection.close()
  print("[❎] Connection closed")
except Exception as e:
  print(f"Error: #{e}")

In the snippet above, the send_to_queue function publishes messages to Lavinmq using the channel.basic_publish() function. You might not really understand the parameters, exchange='', routing_key=routing_key, but at this point do not worry about details like that. Just know that those parameters are used to route these messages to the appropriate queues. Next let’s create our consumer.

Creating the consumer - receiving messages

Here, we will create a basic python application that would receive the plain-text message we will publish and process them.

Again, there are four steps we need to create an application that sends a message to LavinMQ: - First, we need to create a connection to our LavinMQ instance from our Python code - Next, we create a channel within the connection we’ve created - Next, we declare a queue - Then consume messages from the queue

Let’s see what these steps look like in code. First, create a file hello_world_consumer.py in the python_tutorials sub-directory.

The next step would have been to import relevant modules, create a connection and a channel, but the code for doing that hasn’t changed, so feel free to re-use snippets from thevproducer section.

The next step is to declare a queue.

Declare a queue

Add the code snippet below to the end of hello_world_consumer.py file, to declare a queue, named hello_world

channel.queue_declare(queue='hello_world')

Why are we creating the same queue twice -- you ask?

Or maybe you didn’t ask, but still, note that creating a queue with queue_declare() is idempotent - regardless of how many times we run the command, only one queue will be created.

But even more importantly, we are creating the same queue from the consumer side again, just in case you start the consuer first, at which point the hello_world queue declared from the producer side will be non-existent.

Consume messages from the declared queue

The process of receiving messages from the queue is slightly more complex than publishing. It operates by associating a callback function with a queue, and each time a message is received, the Pika library triggers this callback function. In our specific scenario, the function’s purpose is to display the message’s content on the screen.

def callback(ch, method, properties, body):
  print(f"[✅] Received #{ body }")

Next, let’s tie our callback function to LavinMQ and have it receive messages from the hello_world queue

channel.basic_consume(
  "hello_world",
  callback,
  auto_ack=True,
)

We will explain the auto_ack function parameter later.

Lastly, let’s spin up an infinite loop that will allow our consumer to continuously listen for new messages published to lavinMQ and forward messages added to the hello_world queue to our callback function. We will also catch excpetions when they happen.

Users could press CTRL+C or CMD+C(on Mac) to exit this infinite loop.

  try:
    print("\n[❎] Waiting for messages. To exit press CTRL+C \n")
    channel.start_consuming()
  except Exception as e:
    print(f"Error: #{e}")
    try:
      sys.exit(0)
    except SystemExit:
      os._exit(0)

Putting everything together

We’ve worked on the different parts of the producer and consumer in bits, now let’s see what the code would look like when put together.

hello_world_producer.py

# producer.py

import pika, os
from dotenv import load_dotenv

load_dotenv()

# Access the CLOUDAMQP_URL environment variable and parse it (fallback to localhost)
url = os.environ.get('CLOUDAMQP_URL', 'amqp://guest:guest@localhost:5672/%2f')

# Create a connection
params = pika.URLParameters(url)
connection = pika.BlockingConnection(params)
print("[✅] Connection over channel established")

channel = connection.channel() # start a channel
channel.queue_declare(queue="hello_world") # Declare a queue

def send_to_queue(channel, routing_key, body):
  channel.basic_publish(
        exchange='',
        routing_key=routing_key,
        body=body
  )
  print(f"[📥] Message sent to queue #{body}")

# Publish messages
send_to_queue(
    channel=channel, routing_key="hello_world", body="Hello World"
)
send_to_queue(
    channel=channel, routing_key="hello_world", body="Hello World"
)
send_to_queue(
    channel=channel, routing_key="wrong_routing_key", body="Hello World"
)

try:
  connection.close()
  print("[❎] Connection closed")
except Exception as e:
  print(f"Error: #{e}")

hello_world_consumer.py

import pika, os, sys
from dotenv import load_dotenv

load_dotenv()

# Access the CLOUDAMQP_URL environment variable and parse it (fallback to localhost)
url = os.environ.get('CLOUDAMQP_URL', 'amqp://guest:guest@localhost:5672/%2f')

# Create a connection
params = pika.URLParameters(url)
connection = pika.BlockingConnection(params)
print("[✅] Connection over channel established")

channel = connection.channel() # start a channel
channel.queue_declare(queue="hello_world") # Declare a queue

def callback(ch, method, properties, body):
    print(f"[✅] Received #{ body }")

channel.basic_consume(
    "hello_world",
    callback,
    auto_ack=True,
)

try:
  print("\n[❎] Waiting for messages. To exit press CTRL+C \n")
  channel.start_consuming()
except Exception as e:
  print(f"Error: #{e}")
  try:
    sys.exit(0)
  except SystemExit:
    os._exit(0)

Testing our appplications

Spin up two terminals. In the first terminal, run your consumer application with python hello_world_consumer.py

In the second terminal run your producer application with: python hello_world_producer.py

If everything goes well, you should get the following output in the different terminals

Terminal 1 - Consumer

->$ python hello_world_consumer.py
[✅] Connection over channel established

[❎] Waiting for messages. To exit press CTRL+C 

[✅] Received #b'Hello World'
[✅] Received #b'Hello World'

Terminal 2 - Producer

->$ python hello_world_producer.py
[✅] Connection over channel established
[📥] Message sent to queue #Hello World
[📥] Message sent to queue #Hello World
[📥] Message sent to queue #Hello World
[❎] Connection closed

A few things to note

Eeven though we closed connection to the LavinMQ server in the publisher file, in reality, it’s best to keep the connection alive in both the publisher and consumer scripts, rather than opening and closing it repeatedly. This is because opening and closing connections is considered “expensive”.

You might notice that the consumer only received two out of the three messages we sent. Any idea what the bug is? Well, that’s one of your tasks in this section.

Learning lab

Debug the code above and get all three messages to be published to the queue and by extension received by the consumer.
- Hint : The problem is precisely with the line publishing the third message
Optional : What are exchanges, bindings and routing keys
- You don’t have to look into this lab, but learning about these concepts would help you follow the next section more easily.

What’s next?

Next Tutorial: Message routing

Introduction

Configuration

Language Support

AMQP 0-9-1 Overview

More Exchange Types

More Consumer Features

Queue Deep-dive

Other Features

Reliable Message Delivery

High Availability and Backup

Monitoring

Management HTTP API

Tutorials

Networking

LavinMQ CLI

Python hello world

What is LavinMQ

Publishing and consuming messages with Pika, Python client

Python development environment

Creating the producer - sending messages

Creating a connection

Create a channel

Declare a queue

Publish messages to the declared queue

Creating the consumer - receiving messages

Declare a queue

Consume messages from the declared queue

Putting everything together

Testing our appplications

Terminal 1 - Consumer

Terminal 2 - Producer

A few things to note

Learning lab

What’s next?

Managed LavinMQ instance on CloudAMQP

Help and feedback