171–180 of 229 posts
This is a question we asked 6 years ago. What if we wrote it in Rust. And leveraged and WASM. We have been at it for the past 6 years. https://github.com/infinyon/fluvio For the past 2 years we have also been building Flink using Rust and WASM. https://github.com/infinyon/stateful-dataflow-examples/
Any chance you’re going to be reviving support for the Kafka wire protocol?
Earlier quoted context omitted.
Having used both Kafka and Redpanda on several occasions, I'd pick Redpanda any day of the week without a second thought. Easier to setup, easier to maintain, a lot less finicky ans uses a fraction of the resources.
In what way is it materially easier to maintain and less finicky? I read a lot about this but I haven’t seen a concise bullet point list of why, which leads me to naturally distrust such claims. Ditto for the resources - Kafka is usually bottlenecked on disk/network, and whether it’s c++ or not doesn’t solve that
Its a compiled binary - no JVM to manage. Java apps have always been a headache for me. Plus, no zookeeper - one less thing to break.
The biggest benefit I've seen is the performance. Redpanda just outperforms Apache Kakfa on similar hardware. Its also Kafka compliant in every way I've noticed, so all my favorite tools that interact with Kafka work the same with Redpanda.
Redpanda, like Kafka, writes to disk, so you'll always be limited by your hardware no matter what you use (but NVMe's are fast and affordable).
YMMV, but its been a good experience for me.
There are a few interesting projects to replace Kafka: Redpanda / Pulsar / AutoMQ have some of you some experience with those and able to give pros/cons?
Redpanda most importantly is faster that Apache Kafka. We were able to get a lot more throughput. Its also stable, especially compared to dealing with anything that requires a JVM.
Kafka's biggest strength is the wide and useful ecosystem built on top of it.
It is also a weaknesses, as we have to keep some (but not of all) the design decisions we wouldn't have made had we started from scratch today. Or we could drop backwards compatibility, at the cost of having to recreate the ecosystem we already have.
https://nats.io is easier to use than Kafka and already solves several of the points in this post I believe, like removing partitions, supporting key-based streams, and having flexible topic hierarchies.
Earlier quoted context omitted.
Also remember that NATS was donated to the CNCF a while back, and as a result people built a huge ecosystem around it. Easy to forget.
Relevant: https://www.cncf.io/blog/2025/04/24/protecting-nats-and-the-...
https://nats.io is easier to use than Kafka and already solves several of the points in this post I believe, like removing partitions, supporting key-based streams, and having flexible topic hierarchies.
Honestly that website has the least amount of information per text I've seen in multiple websites I had to really dig (outside of that website) to understand even what NATS is and/or does It goes too hard on the keyword babbling and too little on the "what does this actually do" > Services can live anywhere and are easily discoverable - decentralized, zerotrust security Ok cool, this tells me absolutely nothing. What…
Earlier quoted context omitted.
Can you elaborate on how you have “adapted…message causality topologies to cope with consuming mechanisms” in relation to the example of a bank account? The causality topology being what here, couldn’t one day MoneyIn should come before else there can be now true MoneyOut?
Right on, great question. Some examples: Example Option 1 You give up on the guarantees across partition keys (bank accounts), and you accept that balances will not reflect a causally consistent state of the past. E.g., Bob deposits 100, Bob sends 50 to Alice. Balances: Bob 0 Alice 50 # the source system was never in this state Bob 100 Alice 50 # the source system was never in this state Bob 50 Alice 50 # eventually…