Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ursa: A leaderless, object storage–based alternative to Kafka

streamnative.io

Post reply on HN
Print view View original

31–40 of 42 posts

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#31

Post by hubertzhang »

I’m the co-founder of EloqData. I recently gave a talk on EloqDoc and Pulsar integration at Data Streaming Summit 2025. It was a great event, and Ursa was definitely one of the hottest topics discussed.

I believe object storage is shaping the future architecture of cloud databases. The first big shift happened in the data warehouse space, where we saw the move from Teradata and Greenplum to Snowflake accelerate around 2016. Snowflake’s adoption of object storage as its primary storage layer not only reduced costs but also unlocked true elasticity.

Now, we’re seeing a similar trend in the streaming world. If I recall correctly, Ursa was the first to GA an object-storage–based streaming service, with Kafka(WarpStream) and AutoMQ following afterward.

I also believe the next generation of OLTP databases will use object storage as their main storage layer. This blog post shares some insights into this trend and the unique challenges of implementing object storage correctly for OLTP workloads, which are much more latency-sensitive.

https://www.eloqdata.com/blog/2025/07/16/data-substrate-bene...

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#32

Post by upghost »

_benedict wrote: post #26

Do you anywhere elaborate what you mean by leaderless, and how this affects the semantics and guarantees you offer? So far as I understand both Kafka and Pulsar use (leader-based) consensus protocols to deliver some of their features and guarantees, so to match these you must either have developed a leaderless consensus protocol, or modify the guarantees you offer, or else have a leader-based consensus protocol you u…

Whoa a leaderless consensus protocol sounds pretty revolutionary!! So many question -- do you have any resources on this you could share?

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#34

Post by _benedict »

upghost wrote: post #32
_benedict wrote: post #26

Do you anywhere elaborate what you mean by leaderless, and how this affects the semantics and guarantees you offer? So far as I understand both Kafka and Pulsar use (leader-based) consensus protocols to deliver some of their features and guarantees, so to match these you must either have developed a leaderless consensus protocol, or modify the guarantees you offer, or else have a leader-based consensus protocol you u…

Whoa a leaderless consensus protocol sounds pretty revolutionary!! So many question -- do you have any resources on this you could share?

Revolutionary may be an overstatement, it just affords different system characteristics. There's plenty of literature on the topic though, starting generally with EPaxos[1]. The protocol that we are developing is for Apache Cassandra, is called Accord[2], and forms the basis of our new distributed transaction feature [3]. I will note that the whitepaper linked in [3] is a bit out of date, and there was a bug in the protocol specification at that time. We hope to publish an updated paper in a proper venue in the near future.

[1] https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf [2] https://github.com/apache/cassandra-accord [3] https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-15...

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#35

Post by fnordian »

Imustaskforhelp wrote: post #21
sijieg wrote: post #12

Earlier quoted context omitted.

I am one of the co-founders of StreamNative. Currently Ursa is only available in our cloud service. But we do plan to open-source the core soon. Stay tuned.

Can't wait for you guys to open source this stuff. If I may ask, what's the license you guys are thinking of? Since I am interested in hoping to someday live as a developer while working on open source too but its a tough line b/w getting no sponsors with MIT license and being called non foss and being charged in HN for some crimes because you used some license like SSPL or some custom license. The sad reality is tha…

People are suspicious to being rug pulled. There have been many of those instances in the past, where companies advertised with FOSS, but didn’t mean it. A proper FOSS license and clear and assuring communication about the long term freedoms associated with the product are important.

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#36

Post by upghost »

_benedict wrote: post #34
upghost wrote: post #32

Earlier quoted context omitted.

Whoa a leaderless consensus protocol sounds pretty revolutionary!! So many question -- do you have any resources on this you could share?

Revolutionary may be an overstatement, it just affords different system characteristics. There's plenty of literature on the topic though, starting generally with EPaxos[1]. The protocol that we are developing is for Apache Cassandra, is called Accord[2], and forms the basis of our new distributed transaction feature [3]. I will note that the whitepaper linked in [3] is a bit out of date, and there was a bug in the p…

https://www.vldb.org/pvldb/vol15/p1337-lee.pdf

Is this you also or total coincidence?

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#37

Post by _benedict »

upghost wrote: post #36
_benedict wrote: post #34

Earlier quoted context omitted.

Revolutionary may be an overstatement, it just affords different system characteristics. There's plenty of literature on the topic though, starting generally with EPaxos[1]. The protocol that we are developing is for Apache Cassandra, is called Accord[2], and forms the basis of our new distributed transaction feature [3]. I will note that the whitepaper linked in [3] is a bit out of date, and there was a bug in the p…

https://www.vldb.org/pvldb/vol15/p1337-lee.pdf Is this you also or total coincidence?

Not even a coincidence really, it's a very different kind of system. It's an implementation of Hermes with network layer integration. Hermes is designed with very different goals in mind, specifically within-DC consensus with minimal failures (with the caveat I am not intimately familiar):

- Every replica must acknowledge a write, which is undesirable in a WAN setting, due to having to wait for replies from the furthest region

- At most one concurrent "read-modify-write" operation may succeed, so peak throughput is limited by request latency

- Failure of any replica requires reconfiguration for any request to succeed (equivalent to leader election), so the leaderless property here does not improve tail latencies, indeed it is likely harmed by exposing your workload to more required reconfigurations

Cassandra is designed for multiple (usually quite far apart) DC deployments that want to maximise availability and minimise latency, and where failure is expected. Here a quorum system is typically preferable for request latency.

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#38

Post by anchri »

2Elian wrote: post #4

Has anyone tried Ursa before? Curious to hear your thoughts!

I have tried Ursa to export data to Iceberg format in object storage. The real-time data injection of the data was of particular interest in my case as it allows to setup an efficient Lakehouse architecture and enable Snowflake integration without creating a lock-in. Straightforward and efficient.

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#39

Post by chenyang »

hubertzhang wrote: post #31

I’m the co-founder of EloqData. I recently gave a talk on EloqDoc and Pulsar integration at Data Streaming Summit 2025. It was a great event, and Ursa was definitely one of the hottest topics discussed. I believe object storage is shaping the future architecture of cloud databases. The first big shift happened in the data warehouse space, where we saw the move from Teradata and Greenplum to Snowflake accelerate aroun…

[dead]

Re: Ursa: A leaderless, object storage–based alternative to Kafka

#40

Post by BitNomad »

hubertzhang wrote: post #31

I’m the co-founder of EloqData. I recently gave a talk on EloqDoc and Pulsar integration at Data Streaming Summit 2025. It was a great event, and Ursa was definitely one of the hottest topics discussed. I believe object storage is shaping the future architecture of cloud databases. The first big shift happened in the data warehouse space, where we saw the move from Teradata and Greenplum to Snowflake accelerate aroun…

[dead]
Post reply on HN