LavinMQ streams Q&A
Written by: Elin Vinka
Familiarizing with stream queues means adding a Kafka like experience to your AMQP-broker. In LavinMQ stream queues, messages are immutable - as result, they can be read and re-read by multiple consumers. This hands-on tutorial gives you a quick introduction to how they work through a practical use case: streaming user activity data to a stream queue. It’s just one example of how streams can be put to use.
LavinMQ stream queues Q&A
Can I mix traditional messaging and streams in LavinMQ?
Short answer: Yes
Long answer: Stream queue in LavinMQ was not created to replace the traditional queues. Instead, it was made to complement them as they open up new possibilities. In LavinMQ, creating a Stream is as simple as creating any other queue. The only difference is you set the queue type to stream using thex-queue-type argument when you declare it. LavinMQ does not support the creation of non-durable stream queues, meaning you’ll need to set durable=true at declaration time. More information this can be found in the guide.
What is stream filtering and how can it improve performance?
Short answer: Stream filtering reduces unnecessary data transmission, allowing consumers to receive only the messages they need.
Long answer: Stream Filtering (introduced with LavinMQ 2.2) ensures only relevant messages are delivered to consumers. The solution is implemented on the broker. A consumer can pass a single filter value (a string) to the broker or multiple values separated by commas. If a message matches any of the filters, it gets delivered. A consumer could also be configured to receiver all messages without a filter value. Follow the code snippets and illustrations in the guide to get an even deeper understanding of stream filtering.
How does server-side offset tracking work in LavinMQ Streams?
Short answer: Offset tracking acts like a bookmark, helping consumers remember their position in a stream so they can resume processing messages from that point when returning.
Long answer: When consuming from a stream, it is possible **for the consumer to remember where it left off in a stream using a technique called offset tracking.
Offsets are saved on the server for all consumers that meet the following requirements:
- Consumer tag set
- And
x-stream-offset == nullORx-stream-automatic-offset-tracking == true.
The offsets are stored in a separate file, and there’s one file per stream. The stored offsets are replicated to all followers. A new entry is stored for each message delivered, making sure that the tracking is always correct. The file is set to an initial size on startup, and it will be compacted/expanded as needed. In the guide, you can watch offset tracking in action.
Conclusion
Traditional queues in LavinMQ support typical message queuing use-cases where messages are deleted after being consumed. However, some messaging use-cases requires some form of permanent data storage that could easily enable message replay and fan-our architecture, amongst others. Stream queues in LavinMQ were introduced in LavinMQ to cater for such use-cases.
Follow our tutorial series on LavinMQ streams to get started with this queue type.