Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Loro: Reimagine state management with CRDTs

loro.dev

Post reply on HN
Print view View original

31–40 of 45 posts

Re: Loro: Reimagine state management with CRDTs

#31

Post by vosper »

jitl wrote: post #9

The demo code for Loro looks very easy to use, I love how they infer a CRDT from an example plain JS object. I’ve played with a Zod schema Yjs CRDT translator and found it kinda annoying to maintain. However this looks so easy I worry about apps building with too little thought about long term data modeling. Migrations on CRDTs are challenging, so it’s important to “get it right” at the beginning. I’m curious how thi…

> Migrations on CRDTs are challenging, so it’s important to “get it right” at the beginning.

Any tips or dos/don'ts you could share for how to "get it right" at the beginning? I'm hoping to build an application using CRDTs sometime in the future.

Re: Loro: Reimagine state management with CRDTs

#32

Post by allenu »

aatd86 wrote: post #28

Can someone explain to me what happens when there is a destructive update on one side while the other side is still relying on some old version? Can this even be reconciliated? Or is it append only?i.e. No delete operation. UIs have delete operations in general.

I don’t know about the link’s strategy but I’ve implemented something CRDT-like and for object deletion, I’ve had to use tombstones. (In my case I timestamp the tombstone state in case I want to bring the object back).

Re: Loro: Reimagine state management with CRDTs

#33

Post by ablob »

aatd86 wrote: post #28

Can someone explain to me what happens when there is a destructive update on one side while the other side is still relying on some old version? Can this even be reconciliated? Or is it append only?i.e. No delete operation. UIs have delete operations in general.

You can implement delete operations by marking fragments as deleted. That way you can still refer to them from a positional perspective, but they will eventually disappear. The exact behavior depends on the implementation, the only requirement is that every "user" reaches the same state eventually.

  A: delete Line 4@marker
  B: append to the end of line 4@marker "abcd"
could then result in either line 4 with only "abcd", or "abcd" at the beginning of line 5 (as long as both sides resolve to the same state). You're right in that this is tricky to get right, as that is inherent to the complexity of asynchonous mutation from multiple sources.

Re: Loro: Reimagine state management with CRDTs

#34

Post by czx111331 »

jitl wrote: post #9

The demo code for Loro looks very easy to use, I love how they infer a CRDT from an example plain JS object. I’ve played with a Zod schema Yjs CRDT translator and found it kinda annoying to maintain. However this looks so easy I worry about apps building with too little thought about long term data modeling. Migrations on CRDTs are challenging, so it’s important to “get it right” at the beginning. I’m curious how thi…

The code in the blog is from Vue Pinia, a state management library, and not from Loro. It serves as an example demonstrating that CRDTs can be modeled similarly. Thus, you might expect to use Loro in a similar way.

Indeed, schema migration is challenging. There is a lot to explore, both in reducing the need for migration and in ensuring a smooth transition when it's required. We plan to tackle this issue in the future.

Re: Loro: Reimagine state management with CRDTs

#35

Post by tantaman »

hugodutka wrote: post #14
btown wrote: post #11

Earlier quoted context omitted.

What kinds of bugs have you run into? Any large-scale corruption of data at rest?

We have run into queries that corrupted the database client-side, but fortunately that doesn't propagate into postgres itself. In that case we had to drop the client-side db and resync from a clean state. The corruption was also caught by sqlite itself - it threw a "malformed disk image" error and stopped responding to any further queries. Also bugs around syncing some kinds of data - one bug that's already been fixe…

SQLite had 2 bugs[1] where batch atomic writes would corrupt your DB if you used IndexedDB to back your VFS. It has been patched in SQLite so rolling a new electric release that pulls in the latest SQLite build should fix that.

[1] - https://github.com/vlcn-io/js/issues/31#issuecomment-1785296...

Re: Loro: Reimagine state management with CRDTs

#37

Post by enva2712 »

vosper wrote: post #31
jitl wrote: post #9

The demo code for Loro looks very easy to use, I love how they infer a CRDT from an example plain JS object. I’ve played with a Zod schema Yjs CRDT translator and found it kinda annoying to maintain. However this looks so easy I worry about apps building with too little thought about long term data modeling. Migrations on CRDTs are challenging, so it’s important to “get it right” at the beginning. I’m curious how thi…

> Migrations on CRDTs are challenging, so it’s important to “get it right” at the beginning. Any tips or dos/don'ts you could share for how to "get it right" at the beginning? I'm hoping to build an application using CRDTs sometime in the future.

I don’t see why migrations on CRDTs would be categorically different from migrations on other data types, though maybe there’s less open source tooling to leverage at the moment. I’ve got some basic code written on automerge to handle them in a side project

Re: Loro: Reimagine state management with CRDTs

#38

Post by aatd86 »

ablob wrote: post #33
aatd86 wrote: post #28

Can someone explain to me what happens when there is a destructive update on one side while the other side is still relying on some old version? Can this even be reconciliated? Or is it append only?i.e. No delete operation. UIs have delete operations in general.

You can implement delete operations by marking fragments as deleted. That way you can still refer to them from a positional perspective, but they will eventually disappear. The exact behavior depends on the implementation, the only requirement is that every "user" reaches the same state eventually. A: delete Line 4@marker B: append to the end of line 4@marker "abcd" could then result in either line 4 with only "abcd"…

I guess my question is of whether the state that is being reached is a legit one in this case.

What is the source of truth eventually?

I think there must probably be a hierarchy that decides it. It's probably a kind of race condition/byzantine general problem.

Re: Loro: Reimagine state management with CRDTs

#39

Post by jitl »

enva2712 wrote: post #37
vosper wrote: post #31

Earlier quoted context omitted.

> Migrations on CRDTs are challenging, so it’s important to “get it right” at the beginning. Any tips or dos/don'ts you could share for how to "get it right" at the beginning? I'm hoping to build an application using CRDTs sometime in the future.

I don’t see why migrations on CRDTs would be categorically different from migrations on other data types, though maybe there’s less open source tooling to leverage at the moment. I’ve got some basic code written on automerge to handle them in a side project

It’s more challenging because you need to accept updates in format v1 indefinitely even after you apply the v1->v2 migration, which makes it more like maintaining an API with backwards compatibility than the usual SQL migration pattern. For example, let’s say you start with a last-write-wins CRDT for an object of shape { name: string, bio: string } in v1, and then decide bio should use a convergent rich text data type like Peritext. If you do a naive migration and change the type of the field in-place, how do you handle updates from old peers doing a LWW set on bio, when now the data type you expect is a Peritext delta?

And if you’re really peer to peer, you’ll need older peers that don’t understand a new format in their user space software to avoid applying updates that break compatibility for them.

Geoffrey Litt documents the challenges in Cambria, see https://www.inkandswitch.com/cambria/

Re: Loro: Reimagine state management with CRDTs

#40

Post by lann »

aatd86 wrote: post #38
ablob wrote: post #33

Earlier quoted context omitted.

You can implement delete operations by marking fragments as deleted. That way you can still refer to them from a positional perspective, but they will eventually disappear. The exact behavior depends on the implementation, the only requirement is that every "user" reaches the same state eventually. A: delete Line 4@marker B: append to the end of line 4@marker "abcd" could then result in either line 4 with only "abcd"…

I guess my question is of whether the state that is being reached is a legit one in this case. What is the source of truth eventually? I think there must probably be a hierarchy that decides it. It's probably a kind of race condition/byzantine general problem.

The answer depends on the specific CRDT algorithm in use. For complex data structures like the ones behind collaborative text editing your intuition that the updates end up looking hierarchical is generally correct.
Post reply on HN