Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Celld: Self-hosted, distributed Durable Objects

github.com

Post reply on HN
Print view View original

21–30 of 56 posts

Re: Celld: Self-hosted, distributed Durable Objects

#21

Post by kentonv »

rough-sea wrote: post #20
jitl wrote: post #19

what is the difference between celld and Cloudflare Workers open source version workerd? https://github.com/cloudflare/workerd

workerd is missing the scheduler - the thing that allows it to operate across nodes. Durable Object storage are just local sqlite databases. celld is the full distributed system (albeit single tenant). It distributes DOs (cells) across any number of VMs. The databases for each DO are in object storage with RPO=0 guarantee.

You beat us to it! Been meaning to land this but got distracted.

https://github.com/cloudflare/workerd/pull/6780

But honestly I love that there are multiple implementations now.

Re: Celld: Self-hosted, distributed Durable Objects

#23

Post by sakesun »

Wonder if this will become common practice from now on ?

  > Pull requests are disabled. Coding agents make it too easy to send a large, 
  > low-context change that costs maintainers more time than it saves. 
  > Thoughtful contributions are welcome; please understand the code, 
  > keep the patch focused, and respect the review time you are asking for.
  > 
  > Send a git format-patch attachment to ...

Re: Celld: Self-hosted, distributed Durable Objects

#25

Post by jeremycarter »

khalidx wrote: post #11

Finally! So happy to see support for running durable objects outside of one provider. Upvoted. The "durable object" concept has been repeatably demonstrated to be a valuable abstraction. "Each object is its own SQLite database, addressed by name and replicated to an S3-compatible bucket you own" -- this concept can take you a long way, both in its power and simplicity.

This concept has been around for a long time.

https://www.microsoft.com/en-us/research/wp-content/uploads/...

Re: Celld: Self-hosted, distributed Durable Objects

#26

Post by dbmikus »

sakesun wrote: post #23

Wonder if this will become common practice from now on ? > Pull requests are disabled. Coding agents make it too easy to send a large, > low-context change that costs maintainers more time than it saves. > Thoughtful contributions are welcome; please understand the code, > keep the patch focused, and respect the review time you are asking for. > > Send a git format-patch attachment to ...

time to bring back the dev email distribution list

Re: Celld: Self-hosted, distributed Durable Objects

#27

Post by jitl »

kentonv wrote: post #21
rough-sea wrote: post #20

Earlier quoted context omitted.

workerd is missing the scheduler - the thing that allows it to operate across nodes. Durable Object storage are just local sqlite databases. celld is the full distributed system (albeit single tenant). It distributes DOs (cells) across any number of VMs. The databases for each DO are in object storage with RPO=0 guarantee.

You beat us to it! Been meaning to land this but got distracted. https://github.com/cloudflare/workerd/pull/6780 But honestly I love that there are multiple implementations now.

bummer this expects NFS; this is all a brand new implementation vs what Cloudflare runs internally in production for DOs? I can’t imagine all your workerds share a single global NFS… What would a more realistic workerd with DO support look like?

ah well, we have celld!

Re: Celld: Self-hosted, distributed Durable Objects

#28

Post by jitl »

zuzululu wrote: post #8

i donno...i have some doubts about this cloudflare gives you that instant geo-sync across the world that is hard to beat with Celld do i need to buy bare metal in major continents

> cloudflare

the stuff is great when it works but Durable Objects can be quite expensive. whenever i get too excited about em all it takes is a little time trying to price it out to calm me down.

> do i need to buy bare metal

my first idea would be to run celld on AWS Kubernetes deployed to local zones https://docs.aws.amazon.com/eks/latest/userguide/auto-local-...

it’s not “region: earth” like cloudflare but perhaps worth the trade off.

Re: Celld: Self-hosted, distributed Durable Objects

#29

Post by jitl »

q3k wrote: post #7

> addressed by name and replicated to an S3-compatible bucket you own; nodes coordinate through that bucket alone, with no control plane or consensus ... but that S3 _is_ the control plane and consensus layer, no? You're just pushing this down the stack to whoever runs that S3 clone.

sure, but s3 is far easier to buy and also far more reliable than typical distributed systems coordinator stuff like etcd/zookeeper/consul. every major cloud offers it, and there’s several decent open source options.

Re: Celld: Self-hosted, distributed Durable Objects

#30

Post by kentonv »

jitl wrote: post #27
kentonv wrote: post #21

Earlier quoted context omitted.

You beat us to it! Been meaning to land this but got distracted. https://github.com/cloudflare/workerd/pull/6780 But honestly I love that there are multiple implementations now.

bummer this expects NFS; this is all a brand new implementation vs what Cloudflare runs internally in production for DOs? I can’t imagine all your workerds share a single global NFS… What would a more realistic workerd with DO support look like? ah well, we have celld!

Honestly, you wouldn't want to run our internal implementation -- it's far too complicated. Unless you have 100+ datacenters around the world -- then maybe.

The goal of this new design is to scale to a cluster while being operationally very easy to set up. Ideal for self-hosting.

NFSv4 is an easy first step, convenient because it's broadly understood, has many implementations, and requires no client libraries. I could imagine a follow-up to support LiteFS instead of NFS could make a lot of sense, though it'll get more complicated.

But yes, celld is definitely ahead of us here. No doubt about that.

Post reply on HN