For Kafka alternative written in C++ there's Redpanda [1],[2].

Redpanda claim of better performance but benchmarks showed no clear winner [3].

It will be interesting to test them together on the performance benchmarks.

I've got the feeling it's not due to programming language implementation of Scala/Java (Kafka), C++ (Redpanda) and Rust (Walrus).

It's the very architecture of Kafka itself due to the notorious head of line problem (check the top most comments [4].

[1] Redpanda – A Kafka-compatible streaming platform for mission-critical workloads (120 comments):

https://news.ycombinator.com/item?id=25075739

[2] Redpanda website:

https://www.redpanda.com/

[3] Kafka vs. Redpanda performance – do the claims add up? (141 comments):

https://news.ycombinator.com/item?id=35949771

[4] What If We Could Rebuild Kafka from Scratch? (220 comments):

https://news.ycombinator.com/item?id=43790420

As someone who myself worked on a hobby-level Rust based Kafka alternative that used Raft for metadata coordination for ~8 months: nice work!

Wasn't immediately clear to me if the data-plane level replication also happens through Raft or something home-rolled? Getting consistency and reliability right with something home-rolled is challenging.

Notes:

- Would love to see it in an S3-backed mode, either entirely diskless like WarpStream or as tiered storage.

- Love the simplified API. If possible, adding a Kafka compatible API interface is probably worth it to connect to the broader ecosystem.

Best of luck!

• seanhunter

  today at 11:20 AM

  It says on the github page

     " It provides fault-tolerant streaming with automatic leadership rotation, segment-based partitioning, and Raft consensus for metadata coordination."
  

  So I guess that's a "yes" to raft?

  • zbentley

    today at 11:55 AM

    GP asked about data plane consensus, not metadata/control plane.

There's also Iggy https://github.com/apache/iggy

Never tried it, but looks promising

Nice! How does it compare to Redpanda, NATS, etc?