Live data from Hacker News

Ok, but does it scale?

spacetimedb.com

51–60 of 76 posts

Re: Ok, but does it scale?

#51
post #31
post #24

> You get to deploy your server logic directly into the database > You may make use of the Licensed Work provided your application or service uses the Licensed Work with no more than one SpacetimeDB instance in production and provided that you do not use the Licensed Work for a Database Service. Therefore, as an OSS product, SpacetimeDB does not scale.

SpacetimeDB is not Open Source Software. It is "source-available," but is a commercial product.

Interesting, it seems that it's Source Available but converts to Open Source (AGPL) in 2031. First time I've seen licensing like that.

https://github.com/clockworklabs/SpacetimeDB/blob/master/LIC...

Re: Ok, but does it scale?

#52
post #14

The intro section is a good summary of why distributed SQL databases (Spanner, roach, Yugabyte, TiDB) haven't taken off in the market in the same way as say distributed data warehouses have (Snowflake, Databricks, FabricDW, Clickhouse, etc.). I would add a few other things to the list of scaling problems. Some SQL features are hard to scale out (auto_increment/serial columns, unique secondary keys, foreign keys, etc.…

I've pushed teams to choose Spanner over using Postgres (when already in the GCP ecosystem). It's not really more expensive when you sit down and do the math, you save untold hours of maintenance over the lifespan of the app, and it actually scales without fuss.

Is it really not more expensive? I haven't operated a large scale Spanner cluster, but the numbers would suggest it's quite a bit more expensive. I suppose it's workload dependent is what you're saying?

Re: Ok, but does it scale?

#53
post #51
post #31

Earlier quoted context omitted.

SpacetimeDB is not Open Source Software. It is "source-available," but is a commercial product.

Interesting, it seems that it's Source Available but converts to Open Source (AGPL) in 2031. First time I've seen licensing like that. https://github.com/clockworklabs/SpacetimeDB/blob/master/LIC...

ml5.js has a similar license: https://github.com/ml5js/ml5-library?tab=License-1-ov-file

Their license is basically "use freely unless our community decides you violated the code of conduct, for three years"

Re: Ok, but does it scale?

#54

I had a coworker that would always ask this, like a javelin thrown in the bicycle spokes of every demo. We hoped for tens of users.

It depends a bit on context... and scale of scale.

At my first software engineering role the system I 'inherited' (The guy who wrote it quit after it launched, I was hired on after) was only used by 5-10 users at a time. On the flipside, one or two of those users would do things that the system was advertised as being able to do, but would really stress out the system and slow certain operations down for everyone, including exposing our company to financial penalty risks. We actually for a time slowed down trying to get large customers signed up, because even the business folks had to manage enough complaints that they didn't want to risk damage to our reputation. [0]

At my second job, well, I remember the debate I had with the manager about a change I suggested to improve memory allocation on a new email templating process he wrote. He then gave me a lecture about 'premature optimization'. Then when the thing kept crashing with OOM errors he cribbed the parts of my changes that solved the memory leak but left out the pooled buffer to save face. [1]

[0] - The thing was written such that we could only process 2 messages concurrently from a given integration partner at a time, we eventually fixed it by moving the processing to actors behind a consistent hash router.

[1] - The pooled buffer changes were 5-10 extra lines total, lol.

Re: Ok, but does it scale?

#55
post #51
post #31

Earlier quoted context omitted.

SpacetimeDB is not Open Source Software. It is "source-available," but is a commercial product.

Interesting, it seems that it's Source Available but converts to Open Source (AGPL) in 2031. First time I've seen licensing like that. https://github.com/clockworklabs/SpacetimeDB/blob/master/LIC...

The date keeps changing on each release so every version bump becomes AGPL five years from the date its published. Its an interesting model, but necessary to be aware of the nuance in case you ever want to consume it.

Re: Ok, but does it scale?

#57
Off topic but... I had a friend who's main takeaway (tongue-in-check) from harvard mini-mba while he is in a SEAS program was that all you had to do to look smart around VC/tech-oriented business people was to ask this question.

Then we actually founded a startup and realized this was not a silly question. But perhaps that's circular when the default path is VC funding, hmmm.

Re: Ok, but does it scale?

#58

Earlier quoted context omitted.

Fresh reader here. I am very interested to learn more about your sentence > We spent a lot of money finding out that a lock is more performant. I want to hear about that journey. > Besides, it's a btree. I guess it's not a hash table, but I think the point of Vincent's post is still worth exploring. You and he both say that essentially a key/value store is mutexed, and the user's code runs inside that mutex . That is…

I don't have the whole story for you, but the key is that SpacetimeDB transactions are not interactive. The TigerBeetle team talks about this a lot as well. The TL;DR is that because you're not holding locks across the network (as is the case in Postgres), your server code can complete transactions in single digit microseconds, rather than milliseconds. And the practical effect is you can do many more transactions pe…

Would it be correct to say that this is an example of the “Actually Serial Execution” strategy for implementing serializable transactions in the terminology of “Designing Data Intensive Applications”?

(I mention this mainly as a keyword that people can look up for more information)

Re: Ok, but does it scale?

#59

Earlier quoted context omitted.

> It sounds like transactions by default are required to be written to disk before completion They are yolo mode by default with periodic fsync and a big mutex around every reducer: https://strn.cat/posts/spacetime/ (granted things may have changed since that blog post) > I can't recall seeing a network-bound cluster I saw some of these (most packets per second not bandwidth) in the Firebase Realtime Database because…

I concede that we do have a big lock. But that is only because we did the alternative first and it performed worse, which is what OPs article is about. Reposting what I posted below regarding the strn.cat article: I'm a cofounder of SpacetimeDB (and the author of OPs article). The https://strn.cat/posts/spacetime/ article has several substantial errors. I've spoken with Vicent directly about them. Most notably, almos…

What happens if the disk dies?

Re: Ok, but does it scale?

#60

Earlier quoted context omitted.

I concede that we do have a big lock. But that is only because we did the alternative first and it performed worse, which is what OPs article is about. Reposting what I posted below regarding the strn.cat article: I'm a cofounder of SpacetimeDB (and the author of OPs article). The https://strn.cat/posts/spacetime/ article has several substantial errors. I've spoken with Vicent directly about them. Most notably, almos…

What happens if the disk dies?

The SpacetimeDB Cloud version runs with distributed replication, so it continues to be available on the other nodes.

On the SpacetimeDB Standalone (single node) version, you lose your data, same as you would with Postgres or Sqlite.

Post reply on HN