Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ok, but does it scale?

spacetimedb.com

Post reply on HN
Print view View original

41–50 of 76 posts

Re: Ok, but does it scale?

#41

Post by vrosas »

tomxor wrote: post #39
Rohansi wrote: post #28

While I think the tech is cool, and I'm sure it is fast, the fact that the benchmarks make assumptions about your stack doesn't sit right with me. They even admit that it is the main contributor to its speed: > The most significant (but not sole) reason SpacetimeDB is faster than other backends is that we have decreased the round trip time between your server and your database by at least 99.95%. In SpacetimeDB your…

Yes, the irony is that vertically scaling will always go "faster" due to latency until you genuinely need more DB than a single box can handle. Premature horizontal scaling only increase complexity and reduce performance. And even when you genuinely need it the implementation has to fight merely to approach single box latency, which sounds like what these guys have done. But also vertical scaling seems to be able to…

> Premature horizontal scaling

I feel like this isn't a thing when it comes to applications. All the decisions you'd make to allow an app to scale sideways are just, like, good design decisions? Even if you never went 1->2 instances. Otherwise every app would just be PostgREST on top of a beefy VM.

Re: Ok, but does it scale?

#42

Post by timssopomo »

Spacetime sounds like really interesting technology, but I'm not sure that the comparison between CRDB is a good one.

I used to work at Cockroach Labs. The problem it's solving is fundamentally different. CRDB as a solution makes sense when you need to _guarantee_ that transactions are serializable and durable, and that your application can survive node or region failures while maintaining consistency. In a naive deployment it's significantly slower than operating on a single core, but that's the price that you pay for the ability to survive node loss without data loss.

I don't see anything that indicates how spacetime solves the core problem CRDB does, which is guaranteeing that single node failures can be tolerated with zero data loss or loss of availability. It sounds like transactions by default are required to be written to disk before completion, which makes them durable on a single node, but you can't ensure they're consistent across nodes without accepting the network overhead and losing transaction throughput (on writes, anyway).

Also, FWIW, in the several years I worked covering basically every incident, I can't recall seeing a network-bound cluster. Like anything else, there are tradeoffs. You give throughput, you get consistency and availability, and you don't need to engineer how to avoid data loss or availability with node failures. Unless I'm misunderstanding, spacetime is solving a totally different problem.

Re: Ok, but does it scale?

#43

Post by Rohansi »

tomxor wrote: post #39
Rohansi wrote: post #28

While I think the tech is cool, and I'm sure it is fast, the fact that the benchmarks make assumptions about your stack doesn't sit right with me. They even admit that it is the main contributor to its speed: > The most significant (but not sole) reason SpacetimeDB is faster than other backends is that we have decreased the round trip time between your server and your database by at least 99.95%. In SpacetimeDB your…

Yes, the irony is that vertically scaling will always go "faster" due to latency until you genuinely need more DB than a single box can handle. Premature horizontal scaling only increase complexity and reduce performance. And even when you genuinely need it the implementation has to fight merely to approach single box latency, which sounds like what these guys have done. But also vertical scaling seems to be able to…

> I suspect we will eventually end up in a place where the vast majority of applications are happy on a single box, and only twitter scale things need to care about this stuff.

We're already there IMO. A single box can scale up so much more than people expect these days - over a thousand cores and terabytes of RAM. The people coming up with overcomplicated architectures and mandatory high availability get in the way of it.

Re: Ok, but does it scale?

#44

Post by cloutiertyler »

djha-skin wrote: post #37
cloutiertyler wrote: post #29

Earlier quoted context omitted.

We originally did MVCC and it was actually worse performance (in our implementation, I grant), but that's what OPs article is about. We spent a lot of money finding out that a lock is more performant. Calling it a "hashtable" is something that only someone who hasn't built a DB engine would do. It's incredibly naive. It discounts the complexity of execution, atomicity, durability, constraint validation, migrations, q…

Fresh reader here. I am very interested to learn more about your sentence > We spent a lot of money finding out that a lock is more performant. I want to hear about that journey. > Besides, it's a btree. I guess it's not a hash table, but I think the point of Vincent's post is still worth exploring. You and he both say that essentially a key/value store is mutexed, and the user's code runs inside that mutex . That is…

I don't have the whole story for you, but the key is that SpacetimeDB transactions are not interactive. The TigerBeetle team talks about this a lot as well.

The TL;DR is that because you're not holding locks across the network (as is the case in Postgres), your server code can complete transactions in single digit microseconds, rather than milliseconds. And the practical effect is you can do many more transactions per second as a result.

Re: Ok, but does it scale?

#46

Post by cloutiertyler »

convolvatron wrote: post #33
gill-bates wrote: post #4

Earlier quoted context omitted.

I was waiting for a comment like this. It's not like there aren't any projects that should be thinking about "scale", but it's almost a meme at this point. There is even the section there near the beginning about how small web apps without many users 'may have to think about horizontal scaling as well' - no, no they don't. To take part in the meme as well, a single server with Postgres will suffice for what? 99% of c…

its substantially worse than that. anytime anyone uses the word 'scale' without even saying what kind of scaling they mean is guilty of eroding the discourse. at this point I don't think we should even be talking about scaling without meaning 'the Amdahl residual'

Hear hear

Re: Ok, but does it scale?

#47

Post by rockwotj »

timssopomo wrote: post #42

Spacetime sounds like really interesting technology, but I'm not sure that the comparison between CRDB is a good one. I used to work at Cockroach Labs. The problem it's solving is fundamentally different. CRDB as a solution makes sense when you need to _guarantee_ that transactions are serializable and durable, and that your application can survive node or region failures while maintaining consistency. In a naive dep…

> It sounds like transactions by default are required to be written to disk before completion

They are yolo mode by default with periodic fsync and a big mutex around every reducer: https://strn.cat/posts/spacetime/ (granted things may have changed since that blog post)

> I can't recall seeing a network-bound cluster

I saw some of these (most packets per second not bandwidth) in the Firebase Realtime Database because changes get broadcast to many users. Since SpacetimeDB is made for games this is the same synchronization effect. Traditional databases don’t do this which is why Cockroach wouldn’t have seen it.

Re: Ok, but does it scale?

#48

Post by nlawalker »

gill-bates wrote: post #4
randusername wrote: post #3

I had a coworker that would always ask this, like a javelin thrown in the bicycle spokes of every demo. We hoped for tens of users.

I was waiting for a comment like this. It's not like there aren't any projects that should be thinking about "scale", but it's almost a meme at this point. There is even the section there near the beginning about how small web apps without many users 'may have to think about horizontal scaling as well' - no, no they don't. To take part in the meme as well, a single server with Postgres will suffice for what? 99% of c…

Been a meme for a long time!

Mongo DB is web scale

https://www.youtube.com/watch?v=b2F-DItXtZs

Re: Ok, but does it scale?

#49

Post by cloutiertyler »

timssopomo wrote: post #42

Spacetime sounds like really interesting technology, but I'm not sure that the comparison between CRDB is a good one. I used to work at Cockroach Labs. The problem it's solving is fundamentally different. CRDB as a solution makes sense when you need to _guarantee_ that transactions are serializable and durable, and that your application can survive node or region failures while maintaining consistency. In a naive dep…

> which is guaranteeing that single node failures can be tolerated with zero data loss or loss of availability

We solve this with distributed state machine replication. You don't need multiple writers to solve the single node failure problem. You only need multiple writers for a write throughput scaling problem.

They're separate problems.

Re: Ok, but does it scale?

#50

Post by cloutiertyler »

rockwotj wrote: post #47
timssopomo wrote: post #42

Spacetime sounds like really interesting technology, but I'm not sure that the comparison between CRDB is a good one. I used to work at Cockroach Labs. The problem it's solving is fundamentally different. CRDB as a solution makes sense when you need to _guarantee_ that transactions are serializable and durable, and that your application can survive node or region failures while maintaining consistency. In a naive dep…

> It sounds like transactions by default are required to be written to disk before completion They are yolo mode by default with periodic fsync and a big mutex around every reducer: https://strn.cat/posts/spacetime/ (granted things may have changed since that blog post) > I can't recall seeing a network-bound cluster I saw some of these (most packets per second not bandwidth) in the Firebase Realtime Database because…

I concede that we do have a big lock. But that is only because we did the alternative first and it performed worse, which is what OPs article is about.

Reposting what I posted below regarding the strn.cat article:

I'm a cofounder of SpacetimeDB (and the author of OPs article). The https://strn.cat/posts/spacetime/ article has several substantial errors. I've spoken with Vicent directly about them.

Most notably, almost the entire commentary about durability is incorrect. SpacetimeDB does not acknowledge anything before data is fully persisted to disk, even though he claims it does. Clients CAN chose to listen before that, but you can do the same thing in Postgres if you want.

There is no 50 ms delay to writing to disk. The article is mostly nonsense.

Ask Claude yourself: https://github.com/clockworklabs/SpacetimeDB

He spent 15 minutes looking at our code (by his own admission), having never written a database storage engine before AFAIK, and made a pronouncement that SpacetimeDB wasn't a good database. Crazy stuff.

Post reply on HN