Ok, but does it scale? • Hacker News Board

Re: Ok, but does it scale?

#71

Post by Rohansi » Sat, Sep 05, 2026, 2:10 AM UTC

carlm42 wrote: post #68

Rohansi wrote: post #43

Earlier quoted context omitted.

> I suspect we will eventually end up in a place where the vast majority of applications are happy on a single box, and only twitter scale things need to care about this stuff. We're already there IMO. A single box can scale up so much more than people expect these days - over a thousand cores and terabytes of RAM. The people coming up with overcomplicated architectures and mandatory high availability get in the way…

A single box may have that much resources available, but no software will use them efficiently alone. You end up having issues with NUMA, with your pagetable growing significantly forcing you to deal with hugepages and eventually memory fragmentation, and who knows what other kind of lovecraftian horror coming from the sheer complexity of a single system with that much resources. Turns out scaling vertically also has…

For sure, but scaling vertically to that extreme is also unnecessary for the vast majority of systems. You can still have over a hundred cores without needing NUMA. And most systems will never even need that much.

Re: Ok, but does it scale?

#72

Post by andersmurphy » Sat, Sep 05, 2026, 7:45 AM UTC

cloutiertyler wrote: post #60

a2ff6eeb0 wrote: post #59

Earlier quoted context omitted.

What happens if the disk dies?

The SpacetimeDB Cloud version runs with distributed replication, so it continues to be available on the other nodes. On the SpacetimeDB Standalone (single node) version, you lose your data, same as you would with Postgres or Sqlite.

I think people get hung up on high availability (HA), and ironically don't test their backups. Single node with streaming backups to S3 (simple in a single writer system) can give you a system that will at most lose a few seconds of data (interestingly RDS has a 5 minute window of dataloss).

Multi node systems tend to have more failure modes, so fail more often, high availability is not free.

If you replicate data between two nodes (that are note async backups). You have to maintain strong data consistency. So writes to the database are considered successful only if the data is written to both nodes. If either node became unavailable you'd be unable to write anything without sacrificing consistency, reducing our overall availability rather than improving it. Also increasing latency.

This is why the whole multi node HA push by the industry is so laughable. A lot of time it makes systems more fragile.

Re: Ok, but does it scale?

#73

Post by krisajenkins » Sat, Sep 05, 2026, 11:23 AM UTC

gandreani wrote: post #19

Nice! I like learning about databases but got a bit of "framework fatigue" when I started reading about all the number of dbs available today (and deciphering marketing from tech notes). This article hooked me with the comparison at the beginning. I used to scoff at redis' single threaded design but it makes sense in a memory bound db. This article is a great example of taking that high-performance approach and desig…

I totally agree. The thing that's drawing me to Spacetime is that it blends the processing model of actors and the data model of relational databases really neatly.

Data comes in, do something with it, store your state in a principled way, repeat. Add in subscriptions to data and it becomes reactive. Simple.

Re: Ok, but does it scale?

#74

Post by cloutiertyler » Sat, Sep 05, 2026, 2:30 PM UTC

andersmurphy wrote: post #72

cloutiertyler wrote: post #60

Earlier quoted context omitted.

The SpacetimeDB Cloud version runs with distributed replication, so it continues to be available on the other nodes. On the SpacetimeDB Standalone (single node) version, you lose your data, same as you would with Postgres or Sqlite.

I think people get hung up on high availability (HA), and ironically don't test their backups. Single node with streaming backups to S3 (simple in a single writer system) can give you a system that will at most lose a few seconds of data (interestingly RDS has a 5 minute window of dataloss). Multi node systems tend to have more failure modes, so fail more often, high availability is not free. If you replicate data be…

You don’t need CockroachDB for high availability though.

Re: Ok, but does it scale?

#75

Post by cloutiertyler » Sat, Sep 05, 2026, 2:34 PM UTC

rockwotj wrote: post #70

cloutiertyler wrote: post #50

Earlier quoted context omitted.

I concede that we do have a big lock. But that is only because we did the alternative first and it performed worse, which is what OPs article is about. Reposting what I posted below regarding the strn.cat article: I'm a cofounder of SpacetimeDB (and the author of OPs article). The https://strn.cat/posts/spacetime/ article has several substantial errors. I've spoken with Vicent directly about them. Most notably, almos…

Sorry for spreading misinformation!

Hardly your fault! It’s the responsibility of the person writing the article to do the technical diligence. Nevertheless did want to address it.

Re: Ok, but does it scale?

#76

Post by cloutiertyler » Sat, Sep 05, 2026, 2:34 PM UTC

senkora wrote: post #58

cloutiertyler wrote: post #44

Earlier quoted context omitted.

I don't have the whole story for you, but the key is that SpacetimeDB transactions are not interactive. The TigerBeetle team talks about this a lot as well. The TL;DR is that because you're not holding locks across the network (as is the case in Postgres), your server code can complete transactions in single digit microseconds, rather than milliseconds. And the practical effect is you can do many more transactions pe…

Would it be correct to say that this is an example of the “Actually Serial Execution” strategy for implementing serializable transactions in the terminology of “Designing Data Intensive Applications”? (I mention this mainly as a keyword that people can look up for more information)

Yep, that’s exactly what it is.