Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zero-latency SQLite storage in every Durable Object

simonwillison.net

Post reply on HN
Print view View original

81–90 of 108 posts

Re: Zero-latency SQLite storage in every Durable Object

#81

Post by jlokier »

djtango wrote: post #75
jwblackwell wrote: post #72

Does anyone else struggle to wrap their head around a lot of this new cloud stuff? I have 15+ years experience of building for the web, using Laravel / Postgres / Redis stack and I read posts like this and just think, "not for me".

From the article: > For useful background on the first version of Durable Objects take a look at Cloudflare's durable multiplayer moat by Paul Butler, who digs into its popularity for building WebSocket-based realtime collaborative applications. First apps that come to mind that have RT collaboration: - Google Docs/Sheets etc - Notion - Miro - Figma These are all global scale collaborative apps, I'm not sure a Larave…

> Google had to in house everything and probably spearheaded the usage of CRDTs ( this is a guess!)

Fwiw, Google Docs/Sheets etc don't use CRDTs, they use the more server-oriented Operational Transforms (OT). CRDTs were spearheaded by others.

Re: Zero-latency SQLite storage in every Durable Object

#82

Post by 9dev »

I would love to work with Durable Objects and all the other cool stuff from Cloudflare, but I’m really hesitant to make a single cloud providers technology the backbone of my application. If CF decides to pull the plug, or charge a lot more, the only way to migrate elsewhere would be rebuilding the entire app.

As long as there aren’t any comparable technologies, or abstraction layers on top of DOs, I’m not going to make the leap of faith.

Re: Zero-latency SQLite storage in every Durable Object

#84

Post by anentropic »

braden-lk wrote: post #36

Durable objects seem so cool but the pricing always scares me. (Specifically, having to worry about getting hibernation right.) They’d be a great fit for our yjs document based strategy, but while everything in prod still works on plain ol redis and Postgres, it’s hard to justify an exploration.

What scares me is it is super specific to Cloudflare

What is your option if you want to eject to another cloud?

Re: Zero-latency SQLite storage in every Durable Object

#85

Post by emadda »

matharmin wrote: post #65
emadda wrote: post #21

Some other interesting points: - The write api is sync, but it has a hidden async await: when you do your next output with a response, if the write fails the runtime will replace the response with a http failure. This allows the runtime to auto-batch writes and optimistically assume they will succeed, without the user explicitly handling the errors or awaits. - There are no read transactions, which would be useful to…

Just wondering, do you have a specific use case for read transactions implemented on the database level here? In SQLite in general read transactions are useful since you can access the same database from multiple processes at a time. Here, only a single process can access the database. So you can get the same effect as read transactions either by doing all reads in one synchronous function, or implement your own proc…

E.g. if you have many websocket connections and they each have a snapshot at a point in time (that spans over many different await function calls/ws messages).

SQLite can have many readers and a single writer with WAL, so a many read transactions can exist whilst the writers move the db state forward.

Re: Zero-latency SQLite storage in every Durable Object

#86

Post by tlarkworthy »

crabmusket wrote: post #61
tlarkworthy wrote: post #58

Earlier quoted context omitted.

It's the actor model essentially. You can have a DO proxy each user connection, then they forward messages to the multipler document. The user proxy deals with ordering and buffering their connection message state in the presence of disconnects, and the document DO handles the shared state.

It's actors plus a global routing system that means all messages addressed to a unique identifier will arrive in the actor instance. I haven't seen any other actor frameworks that provide that.

Akka and Erlang both support distributed routing to their actors, but this is planetary scale and fully-managed out of the box, which is very cool.

Re: Zero-latency SQLite storage in every Durable Object

#88

Post by klabb3 »

crabmusket wrote: post #28
klabb3 wrote: post #25

Earlier quoted context omitted.

Databases is an extremely slow-maturing area, similar to programming languages, but are all deviations from Postgres shiny and hipster? The idea of colocating data and behavior is really a quantifiable reduction in complexity. It removes latency and bandwidth concerns, which means both operational concerns and development concerns (famously the impact of the N+1 problem is greatly reduced). You can absolutely argue t…

> That gives noSQL nightmares - breaking all kinds of important invariants of relational dbs IMO Durable Objects map well to use cases where there actually are documents. Think of Figma. There is a ton of data that lives inside the literal Figma document. It would be awful to have a relational table for like "shapes" with one row per rectangle across Figma's entire customer base. That's just not an appropriate use of…

> Durable Objects map well to use cases where there actually are documents

Right. I wouldn’t dispute this. This is akin to a file format from software back in the day (like say photoshop but now with multiplayer). What this means is that you get different compatibility boundaries and you relinquish centralized control and ability to do transparent migrations and analysis. For all intents and purposes, the documents should be more or less opaque and self-contained. I personally like this, but I also recognize that most web engineers of our current generation are not used to think in this disciplined and defensive way upfront.

Re: Zero-latency SQLite storage in every Durable Object

#89

Post by esnard »

vlaaad wrote: post #87

Re https://where.durableobjects.live/ — why the hell are they still operating in Russia?

From https://blog.cloudflare.com/steps-taken-around-cloudflares-s... :

> Since the invasion, providing any services in Russia is understandably fraught. Governments have been united in imposing a stream of new sanctions and there have even been some calls to disconnect Russia from the global Internet. As discussed by ICANN, the Internet Society, the Electronic Frontier Foundation, and Techdirt, among others, the consequences of such a shutdown would be profound.

> [...]

> Beyond this, we have received several calls to terminate all of Cloudflare's services inside Russia. We have carefully considered these requests and discussed them with government and civil society experts. Our conclusion, in consultation with those experts, is that Russia needs more Internet access, not less.

Post reply on HN