<- Back to blog archive
Engineering Crystal

LavinMQ – Engineering the bare minimum

LavinMQ – Engineering the bare minimum

We engineered LavinMQ from first principles to answer one question: What is the fastest possible way to move data from a publisher to a consumer? Or, more specifically: How to maximize throughput and resource efficiency in message queueing and streaming.

At LavinMQ, we focused on a specific goal: streamlining the journey of data from the wire to the disk. To achieve this, we collapsed the data path to its bare minimum, bridging the gap between RAM and disk using memory-mapped files (mmap). This allows the application and the operating system to share the same memory space for data storage. Normally, saving data requires system calls (syscall) to transition data from the application to the kernel. These calls involve context switches that can naturally limit performance.

Keeping the I/O path lean and fast

Instead of issuing multiple read and write calls, we bypass this overhead by mapping the message log directly into our process’s address space. When data arrives, we perform a single memcpy from the network socket directly into this contiguous memory block.

By offloading the heavy lifting to the operating system’s kernel, which manages disk synchronization in the background through demand paging. It creates a direct, high-speed path: the producer writes to a memory address, and the consumer reads from that same location. The result is zero-copy access and inherent safety; because the data is already in the hands of the OS, it remains ready for persistence even if the application crashes. This approach pushes message throughput and latency toward hardware limits while still providing persistence and crash recovery.

When building fast software, choose a fast language

To keep this path clean, we chose Crystal. It’s a language that compiles to native code, giving us direct, low-level control over sockets and memory. There’s no heavy runtime or virtual machine between the broker and the operating system. We also love Crystal’s simple syntax.

Less is more

We didn’t just build a faster broker; we built a more predictable and resource-efficient one. When you engineer for the bare minimum, there are fewer places for bugs to hide, fewer “moving parts” to fail, and significantly less overhead to pay for in your cloud bill.

By focusing on a collapsed data path built on Crystal and mmap-backed logs, we achieved our goal: a lightweight, high-performance broker that works in harmony with the hardware. LavinMQ is proof that simplicity can be the most effective way to solve complex problems. Sometimes, the most sophisticated solution is simply getting out of the way of the hardware.