Introduction
Configuration
Language Support
AMQP 0-9-1 Overview
More Exchange Types
More Consumer Features
Queue Deep-dive
Other Features
Reliable Message Delivery
High Availability and Backup
Monitoring
Management HTTP API
Tutorials
Networking
LavinMQ CLI
Ruby hello world
This part is also available in: Python Node.js
You might be familiar with what LavinMQ is already. But to reiterate, we are going to go over it again, but this time around, in an approachable manner.
What is LavinMQ
LavinMQ is a message broker also called a message queue - But what is a message broker?
From a non-technical standpoint, a broker is literally a middle man that facilitates a transaction between two parties, usually a buyer and a seller. Message brokers do pretty much the same thing in the world of software.
LavinMQ and other brokers like it, facilitate the transfer of binary blobs of data, known as messages, between a sending application and a receiving application interested in the message. But how?
-
The sending application, known as a producer, creates and sends messages to LavinMQ. This process is referred to as publishing.
- As a message broker, LavinMQ receives and securely stores messages from producers until
they are retrieved by receiving applications. LavinMQ utilizes its built-in queue that
maintains the order of arrival and persists messages until they are picked up.
- To better get a sense of how a qeueue works in LavinMQ and in fact in most message brokers, you can think of it as a cylinder with open ends. New messages are added to the queue from one end of the cylinder and only removed from the opposite end - First In First Out(FIFO).
- A receiving application formally called the consumer receives messages from LavinMQ and processes them at its own pace. This process is referred to as consuming.
The image below visualizes the flow of messages from the Producer -> LavinMQ -> Consumer.
Some noteworthy information on producers, consumers and messages: - Producers and consumers can be processes running on the same computer, modules of an application, or services running on different computers with varying technology stacks. - Messages are the data added by the sender application to the queue. They can include task requests for the receiving system, like sending an email or resizing an image, as well as plain text or information about completed tasks.
Now that we get the basics of LavinMQ, let’s see how we can set up a basic producer that publishes messages to LavinMQ and a corresponding consumer that consumes the messages.
Publishing and consuming messages with amqp-client, Ruby client
Ruby developers have a number of options for AMQP client libraries. In this tutorial, AMQP::Client is used, an asynchronous client for publishing and consuming of messages.
We will build the simplest thing you can make with LavinMQ - two rudimentary Ruby applications. The first Ruby application, the producer will publish messages to LavinMQ and the second application, the consumer will receive these messages from LavinMQ.
There are three steps we will take to build our rudimentary producer and consumer: 1. Setting up the Ruby development environment - we will only have this step in this section. For subsequent Ruby tutorials, we will reference this guide and have you replicate steps where necessary. 1. Creating the producer 1. Creating the consumer
Ruby development environment
- First make sure that you have Ruby and that it’s available from your command line.
You can confirm this by running:
ruby --version
- If you do not have Ruby installed, go to ruby-lang.org or you could even use rbenv
to manage multiple Ruby versions - Create a directory for your LavinMQ tutotirls if it does not exist already. Run
mkdir lavinmq_tutorials && cd lavinmq_tutorials
to create a directory named lavinmq_tutorials and then navigate into it - this directory will house the tutorial in this section and all the subsequent ones. We will create sub-directories in this folder to organize the different tutorials based on technologies. - next create a sub-directory for the Ruby tutorials in the
lavinmq_tutorials
directory. Runmkdir ruby_tutorials && cd ruby_tutorials
to create a sub directory named ruby_tutorials and then navigate into it - Open your project directory in your favourite text-editor and create a
Gemfile
in theruby_tutorials
sub-directory. Add the following gems to the Gemfile:source 'https://rubygems.org' gem 'amqp-client' gem 'dotenv'
- Install gems with bundler
bundle install
- running this command should generate aGemfile.lock
file in theruby_tutorials
sub-directory - If you, however, do not have bundler installed, go ahead and install it with
gem install bundler
. Alternatively, you can install each gem seperately withgem install gem-name
- Create a
.env
file in in the root directory - Add
CLOUDAMQP_URL="lavinmq_url"
to the `.env’ file. Replace lavinmq_url with your correct server url
Creating the producer - sending messages
Here, we will create a basic Ruby application that publishes a plain text message to LavinMQ - the hello world of LavinMQ.
There are four steps we need to create an application that sends a message to LavinMQ: - First, we need to create a connection to our LavinMQ instance from our Ruby code - Next, we create a channel within the connection we’ve created - Next, we declare a queue - Then publish messages to the queue
Let’s see what these steps look like in code
Creating a connection
Create a file hello_world_producer.rb
in the ruby_tutorials
sub-directory. and add the snippet below to the file:
require "amqp-client"
require "dotenv/load"
# Grab lavinmq url from .env file
lavinmq_url = ENV['CLOUDAMQP_URL']
In the snippet above, we imported relevant packages and loaded our Lavinmq
connection string from the .env
file. Next, let’s create a connection to our Lavinmq
server. Add the code snippet below to the end of hello_world_producer.rb
file.
# Opens and establishes a connection
connection = AMQP::Client.new(lavinmq_url).connect
Create a channel
Add the code snippet below to the end of hello_world_producer.rb
file, to create a
channel within the connection we’ve created in the previous step.
channel = connection.channel()
puts "[✅] Channel over a connection created"
Now that we have a connection and a channel, let’s try to make sense of what they are.
If we think of sending a message from a producer to a Lavinmq instance in terms of, let’s say, delivering a package between two houses in a neighbourhood, then we can say a connection is the direct road linking these two houses and a channel, the multiple paths within that road.
In more technical terms, a connection is the direct link between a producer or consumer and a Lavinmq server, giving the producer or consumer access to the Lavinmq server.
Because you are most likely going to be needing to send or receive multiple messages, a channel allows the re-use of a connection - think of channels as smaller connections within a connection - all messages are published and consumed on a channel.
Declare a queue
Add the code snippet below to the end of hello_world_producer.rb
file, to declare a
queue, named hello_world
channel.queue_declare(queue='hello_world')
hello_world
in the snippet above is the queue name. In the event that you
do not pass a queue name, one would be auto-generated.
Publish messages to the declared queue
We are going to create a re-usable function that we can use to publish messages
to LavinMQ. Add the code snippet below to the end of hello_world_producer.rb
file,
to create a function for publishing messages, publish three messages, “Hello World” and
close the connection.
def send_to_queue(channel, routing_key, body)
# Publish function expects: body, exchange, routing_key in that order
channel.basic_publish(
body
'',
routing_key
)
puts "[📥] Message sent to queue - msg: #{body}"
end
# Publish messages
send_to_queue channel, "hello_world", "Hello World - 1"
send_to_queue channel, "hello_world", "Hello World - 2"
send_to_queue channel, "wrong_routing_key", "Hello World - 3"
begin
connection.close
puts "[❎] Connection closed"
rescue => exception
puts "Error: #{exception}"
end
In the snippet above, the send_to_queue
function publishes messages to Lavinmq
using the channel.basic_publish()
function.
body
is the message that will be published to the queue, the empty string passed, stands in place of an exchange name that should have been passed to
the function. And the routing_key
, well, is the routing key.
At this point, exchanges and routing keys might not make so much sense to you,
but do not worry about details like that. Just know that those parameters are used
to route these messages to the appropriate queues. Next, let’s create our consumer.
Creating the consumer - receiving messages
Here, we will create a basic Ruby application that would receive the plain-text message we will publish and process them.
Again, there are four steps we need to create an application that sends a message to LavinMQ: - First, we need to create a connection to our LavinMQ instance from our Ruby code - Next, we create a channel within the connection we’ve created - Next, we declare a queue - Then consume messages from the queue
Let’s see what these steps look like in code. First, create a
file hello_world_consumer.rb
in the ruby_tutorials
sub-directory.
The next step would have been to import relevant modules, create a connection and a channel, but the code for doing that hasn’t changed, so feel free to re-use snippets from the producer section.
The next step is to declare a queue.
Declare a queue
Add the code snippet below to the end of hello_world_consumer.rb
file, to declare a
queue, named hello_world
channel.queue_declare(queue='hello_world')
Why are we creating the same queue twice -- you ask?
Or maybe you didn’t ask, but still, note that creating a queue with queue_declare()
is idempotent - regardless of how many times we run the
command, only one queue will be created.
But even more importantly, we are creating the same queue from the consumer
side again, just in case you start the consumer first, at which point the hello_world
queue declared from the producer side will be non-existent.
Consume messages from the declared queue
Add the code snippet below to the end of hello_world_consumer.rb
file,
to subscribe to the queue we’ve declared and have our consumer continuously
listen and receive messages published to that queue.
counter = 0
# Subscribe to the queue
channel.basic_consume("hello_world") do |msg|
counter += 1
# Add logic to handle the message here...
puts "[📤] Message received [#{counter}]: #{msg.body}"
end
# Close the connection when the script exits
at_exit do
client.stop
puts "[❎] Connection closed"
end
# Keep the consumer running
sleep
The channel.basic_consume("hello_world")
do |msg| block is now receiving messages from the queue, in an attempt to process the information. This is where whatever
custom logic one decides to work on could be added - - in our case, we just
log the messages received to the console
sleep
, in the snippet above will keep the consumer running infinitely.
Users could press CTRL+C or CMD+C(on Mac) to exit this infinite loop.
Putting everything together
We’ve worked on the different parts of the producer and consumer in bits, now let’s see what the code would look like when put together.
hello_world_producer.py
require "amqp-client"
require "dotenv/load"
# Grab lavinmq url from .env file
lavinmq_url = ENV['CLOUDAMQP_URL']
# Opens and establishes a connection
connection = AMQP::Client.new(lavinmq_url).connect
# Open a channel
channel = connection.channel
puts "[✅] Connection over channel established"
# Create a queue
queue = channel.queue_declare("hello_world")
def send_to_queue(channel, routing_key, body)
# Publish function expects: body, exchange, routing_key in that order
channel.basic_publish(
body,
'',
routing_key
)
puts "[📥] Message sent to queue - msg: #{body}"
end
# Publish messages
send_to_queue channel, "hello_world", "Hello World - 1"
send_to_queue channel, "hello_world", "Hello World - 2"
send_to_queue channel, "wrong_routing_key", "Hello World - 3"
begin
connection.close
puts "[❎] Connection closed"
rescue => exception
puts "Error: #{exception}"
end
hello_world_consumer.rb
require "amqp-client"
require "dotenv/load"
# Grab lavinmq url from .env file
lavinmq_url = ENV['CLOUDAMQP_URL']
# Opens and establishes a connection
connection = AMQP::Client.new(lavinmq_url).connect
# Open a channel
channel = connection.channel
puts "[✅] Connection over channel established"
puts "[❎] Waiting for messages. To exit press CTRL+C "
# Create a queue
queue = channel.queue_declare("hello_world")
counter = 0
# Subscribe to the queue
channel.basic_consume("hello_world") do |msg|
counter += 1
# Add logic to handle the message here...
puts "[📤] Message received [#{counter}]: #{msg.body}"
end
# Close the connection when the script exits
at_exit do
client.stop
puts "[❎] Connection closed"
end
# Keep the consumer running
sleep
Testing our applications
Spin up two terminals. In the first terminal, run your consumer application
with ruby hello_world_consumer.rb
In the second terminal run your producer application with:
rub hello_world_producer.rb
If everything goes well, your output should look like what’s shown in the image below
Terminal 1 - Consumer
->$ ruby hello_world_consumer.rb
[✅] Connection over channel established
[❎] Waiting for messages. To exit press CTRL+C
[📤] Message received [1]: Hello World - 1
[📤] Message received [2]: Hello World - 2
Terminal 2 - Producer
->$ ruby hello_world_producer.rb
[✅] Connection over channel established
[📥] Message sent to queue - msg: Hello World - 1
[📥] Message sent to queue - msg: Hello World - 2
[📥] Message sent to queue - msg: Hello World - 3
[❎] Connection closed
A few things to note
Eeven though we closed connection to the LavinMQ server in the publisher file, in reality, it’s best to keep the connection alive in both the publisher and consumer scripts, rather than opening and closing it repeatedly. This is because opening and closing connections is considered “expensive”.
You might notice that the consumer only received two out of the three messages we sent. Any idea what the bug is? Well, that’s one of your tasks in this section.
Learning lab
- Debug the code above and get all three messages to be published to the queue
and by extension received by the consumer.
- Hint : The problem is precisely with the line publishing the third message
- Optional : What are exchanges, bindings and routing keys
- You don’t have to look into this lab, but learning about these concepts would help you follow the next section more easily.
What’s next?
- Next Tutorial: Message routing
Ready to take the next steps? Here are some things you should keep in mind:
Managed LavinMQ instance on CloudAMQP
LavinMQ has been built with performance and ease of use in mind - we've benchmarked a throughput of about 1,000,000 messages/sec. You can try LavinMQ without any installation hassle by creating a free instance on CloudAMQP. Signing up is a breeze.
Help and feedback
We welcome your feedback and are eager to address any questions you may have about this piece or using LavinMQ. Join our Slack channel to connect with us directly. You can also find LavinMQ on GitHub.