Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zero-latency SQLite storage in every Durable Object

simonwillison.net

Post reply on HN
Print view View original

61–70 of 108 posts

Re: Zero-latency SQLite storage in every Durable Object

#61

Post by crabmusket »

tlarkworthy wrote: post #58
crabmusket wrote: post #27

Earlier quoted context omitted.

Best case, the players are co-located in a city or country, and they'll benefit from data center locality. Worst case, they're not co-located, and one participant has good latency, and the other doesn't. This is equivalent to the "deploy the backend in a single server/datacenter" approach. Aside from the data locality, I still find the programming model (a globally-unique and addressable single-threaded class instanc…

It's the actor model essentially. You can have a DO proxy each user connection, then they forward messages to the multipler document. The user proxy deals with ordering and buffering their connection message state in the presence of disconnects, and the document DO handles the shared state.

It's actors plus a global routing system that means all messages addressed to a unique identifier will arrive in the actor instance. I haven't seen any other actor frameworks that provide that.

Re: Zero-latency SQLite storage in every Durable Object

#63

Post by memothon »

tmikaeld wrote: post #57

> ..each DO constantly streams a sequence of WAL entries to object storage - batched every 16MB or every ten seconds. Which also means it may take 10 seconds before you can (reliably) read the write globally. I keep failing to see how this can replace regionally placed database clusters which can serve a continent in milliseconds. Edit: I know it uses streams, but those are only to 5 followers and CF have hundreds of…

Those WAL entries streamed to object storage I think are just for backups.

Each DO is globally unique (there's one DO with a given id running anywhere) and runs sqlite on its own local storage in that datacenter.

Re: Zero-latency SQLite storage in every Durable Object

#64

Post by pests »

masterj wrote: post #9
simonw wrote: post #6

One thing I don't understand about Durable Objects yet is where they are physically located. Are they located in the region that hosted the API call that caused them to be created in the first place? If so, is there a mechanism by which a DO can be automatically migrated to another location if it turns out that e.g. they were created in North America but actually all of the subsequent read/write traffic to them comes…

> Durable Objects do not currently change locations after they are created > Dynamic relocation of existing Durable Objects is planned for the future. https://developers.cloudflare.com/durable-objects/reference/... . IIRC Orleans ( https://www.microsoft.com/en-us/research/wp-content/uploads/... ) allows actors to be moved between machines, which should map well to DOs being moved between locations.

As actors in Orleans are virtual and persistent it can also be the case it is running nowhere.

If it's stateless it could be running in multiple locations.

I worry "Dynamic relocation of DOs" might be going a bit too granular, this should be something the runtime takes care of.

Re: Zero-latency SQLite storage in every Durable Object

#65

Post by matharmin »

emadda wrote: post #21

Some other interesting points: - The write api is sync, but it has a hidden async await: when you do your next output with a response, if the write fails the runtime will replace the response with a http failure. This allows the runtime to auto-batch writes and optimistically assume they will succeed, without the user explicitly handling the errors or awaits. - There are no read transactions, which would be useful to…

Just wondering, do you have a specific use case for read transactions implemented on the database level here?

In SQLite in general read transactions are useful since you can access the same database from multiple processes at a time. Here, only a single process can access the database. So you can get the same effect as read transactions either by doing all reads in one synchronous function, or implement your own process-level locking.

Re: Zero-latency SQLite storage in every Durable Object

#66

Post by rozenmd »

segalord wrote: post #62

Noticing CF pushing for devs to use DO for eveything over workers these days. Even websocket connections on workers get timed out after ~30s and the recommended way is to use DO for them

Durable Objects have always been the recommended way to do websocket connections on Cloudflare Workers? (as far as I remember, anyway)

The original chat demo dates back to 2020, using DOs + websockets: https://github.com/cloudflare/workers-chat-demo

Re: Zero-latency SQLite storage in every Durable Object

#67

Post by _1tem »

What I don’t understand is why, in the example of flight seat mapping provided, you create a DO per flight. So does a DO correspond to a “model” in MVC architecture? What if I used DOs in a per-tenant way, so one DO per user. And then how do I query or “join” across all DOs to find all full flights? I guess you would have to design your DOs such that joins are not required?

Re: Zero-latency SQLite storage in every Durable Object

#68

Post by firtoz »

tmikaeld wrote: post #57

> ..each DO constantly streams a sequence of WAL entries to object storage - batched every 16MB or every ten seconds. Which also means it may take 10 seconds before you can (reliably) read the write globally. I keep failing to see how this can replace regionally placed database clusters which can serve a continent in milliseconds. Edit: I know it uses streams, but those are only to 5 followers and CF have hundreds of…

AFAIK the writes and reads are done only from the same process, so the long term storage will apply only if the current process is hibernated. When you write something and then read it, it's immediate, because the writes and reads are also updating the current process's state in memory.

For another process (e.g. another DO or another worker) to access the data, they need to go through the DO which "contains" the data, so they'd be making a RPC or a HTTP request to the DO, and they'd get the latest information.

+ the hibernation happens after x seconds of inactivity, so it feels like the only time a data write to be unavailable as expected would be when the DO or worker crashes right after a write.

Re: Zero-latency SQLite storage in every Durable Object

#69

Post by skrebbel »

I really love the Durable Object design, particularly because it's easy to understand how it works on the inside. Unlike lots of other solutions designed for realtime data stuff, Durable Objects have a simplicity to them, much like Redis and Italian food. You can see all the ingredients. Given enough time and resources (and datacenters :) ), a competent programmer could read the DO docs and reimplement something similar. This makes it easy to judge the tradeoffs involved.

I do worry that DOs are great for building fast, low-overhead, realtime experiences (eg five people editing a document in realtime), but make it very hard to make analyses and overviews (which groups of people have been which editing documents the last week?). Putting the data inside SQLite might make that even harder - you'd have to somehow query lots and lots of little SQLite instances and then merge the results together. I wonder if there's anything for this with DOs, because this is what keeps bringing me back to Postgres time and time again: it works for core app features and for overviews, BI, etc.

Re: Zero-latency SQLite storage in every Durable Object

#70

Post by tmikaeld »

firtoz wrote: post #68
tmikaeld wrote: post #57

> ..each DO constantly streams a sequence of WAL entries to object storage - batched every 16MB or every ten seconds. Which also means it may take 10 seconds before you can (reliably) read the write globally. I keep failing to see how this can replace regionally placed database clusters which can serve a continent in milliseconds. Edit: I know it uses streams, but those are only to 5 followers and CF have hundreds of…

AFAIK the writes and reads are done only from the same process, so the long term storage will apply only if the current process is hibernated. When you write something and then read it, it's immediate, because the writes and reads are also updating the current process's state in memory. For another process (e.g. another DO or another worker) to access the data, they need to go through the DO which "contains" the data…

You're right that reads and writes are immediate in the same client connection, this is how it works with CF KV as well - but not across the entire network.

On KV they expect up to 30 second latency before a write can be written everywhere, I expect similar here.

Post reply on HN