Introduction
Configuration
Language Support
AMQP 0-9-1 Overview
More Exchange Types
More Consumer Features
Queue Deep-dive
Other Features
Reliable Message Delivery
High Availability
Monitoring
Management HTTP API
Tutorials
Networking
LavinMQ CLI
LavinMQ Streams guide
Imagine you need to track user activity in your application: clicks, page views, searches, and other events. The application needs to stream these events to a message broker in real time. On the receiving end, several services consume and process these events:
- A real-time analytics service: Updates dashboards with user behaviour trends.
- A recommendation engine: Tailors recommendations based on user activity.
- A session analysis service: Aggregates user activity into cohesive user sessions.
This use-case requires the message broker to handle:
- High throughput: Thousands or even millions of user events per second, streaming to the broker.
- Multiple consumers: Each service consuming the same event stream to perform its specific function.
- Replay and time-travel: Occasionally, a service might need to reprocess older events, either for debugging or recovering from a failure.
AMQP-based brokers like LavinMQ were built to cater to conventional messaging use cases: a producer sends a message to a queue, a consumer retrieves the message, acknowledges it, and the broker removes it from the queue. While this model works well for many scenarios, it is not ideal for use-cases requiring fan-out architectures and replay capabilities — Apache Kafka and other tools like it, have been the go-to tools for use-cases like this.
The Message Queueing landscape is changing: Enter streams
The Stream queue was introduced in LavinMQ to expand the horizons of what can be accomplished with this broker. Unlike the normal queues in LavinMQ, Stream Queues are immutable — messages are not deleted after being consumed. This allows multiple consumers to read the same message and enables replaying old messages as well. Stream queues provide a Kafka-like experience within the LavinMQ ecosystem.
What to expect in this guide
The objective of this guide is simple: to help you learn about and work with LavinMQ Streams. In the end, you will have an understanding of Streams and how to use them in event-streaming scenarios.