Show HN: Walrus – a Kafka alternative written in Rust

Rust-made Walrus takes on Kafka — fans hype, veterans ask hard questions

TLDR: Walrus is a new Rust-based system aiming to rival Kafka with simple setup, rotating leaders, and Raft-managed metadata. Commenters applaud the build but demand real benchmarks and worry about custom data replication, while jokes fly, making this a hype-versus-proof moment worth watching.

Rust devs just splashed a new beast into the data river: Walrus, a Rust-built engine that moves messages around like a superfast mailroom. It promises crash-resistance via Raft (a way computers agree on who’s in charge), auto leader rotation to spread traffic, sealed segments for easy history reads, and a simple text-over-TCP client. There’s even Linux speed magic under the hood with io_uring. Sounds slick — but the comments turned into a spicy town hall.

The top vibe: “Cool project, but prove it.” One veteran cheered the build and immediately side-eyed the data replication story, asking if it’s Raft or “home-rolled,” warning that rolling your own consistency is a pain. Another voice went full reality check: performance isn’t about Rust vs Java vs C++; benchmarks showed no clear winner between rivals like Redpanda, so show us numbers. Curious onlookers kept asking how Walrus stacks up against Redpanda, NATS, and even dropped an alternative: Iggy.

Meanwhile, the comic relief stole the thread: a dev once tried their own Kafka challenger named “tuberculosis” — because that’s what killed Kafka — and the crowd lost it. The mood is equal parts hype and audit. People love the easy 3‑node setup and friendly CLI, but they want S3 support, hard benchmarks, and clarity on data-plane replication before declaring Walrus king.

Key Points

  • Walrus is a distributed message streaming engine with segment-based partitioning, automatic leadership rotation, and fault-tolerant metadata coordination via Raft.
  • Producers/consumers connect to any node; the cluster routes to segment leaders and manages segment rollovers to distribute load.
  • Each node includes Node Controller, Raft Engine (Octopii), Cluster Metadata, and Bucket Storage, with leases synced every 100ms to fence writes.
  • Storage uses WAL files and io_uring on Linux; sealed segments enable reads from any replica without data movement.
  • A simple TCP client protocol supports REGISTER, PUT, GET, STATE, and METRICS commands, with JSON responses for metadata and metrics; quick-start and configuration flags are provided for a 3-node cluster.

Hottest takes

"Getting consistency and reliability right with something home-rolled is challenging." — roncohen
"benchmarks showed no clear winner" — teleforce
"We called it `tuberculosis`… of course, that is what killed Kafka." — lionkor
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.