Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Loro's rich text CRDT

loro.dev

Post reply on HN
Print view View original

41–49 of 49 posts

Re: Loro's rich text CRDT

#41

Post by czx111331 »

kiitos wrote: post #29
jitl wrote: post #6

I'm curious about this line describing REG: > The REG algorithm excels with its fast local update speeds and eliminate concerns about tombstone collection in CRDTs. For instance, if an operation has been synchronized across all endpoints, no new operations will occur concurrently with it, allowing it to be safely removed from the history. If you remove these ops from history, does that remove the ability to time trav…

> if an operation has been synchronized across all endpoints, no new operations will occur concurrently with it, allowing it to be safely removed from the history. This assumes that the set of endpoints (really, nodes) is both well-known by all other nodes in the network, and stable over time (meaning new nodes will never be added). Even if this assumption can be made safely (which is not a given) the GC process desc…

Similar to OT, in certain scenarios, it's sufficient to ensure that only a subset of peers have the complete data, while others don't need the full history. For instance, in real-time collaboration scenarios with a central server, we can, just like OT, allow clients to hold only a shallow clone instead of the complete history. This approach results in minimal overhead for the clients.

Re: Loro's rich text CRDT

#42

Post by czx111331 »

mikebelanger wrote: post #32

Looks neat! Would there be a way of intercepting state and making 'snapshots' into a more traditional format, like SQL, or even a JSON file? It sounds like this defaults to the server storing the whole state in their binary format, ditto the client-side portion of it. Nothing wrong with the format, but this is an early project, and nobody wants their data in something that's potentially unstable, or something that mi…

We are carefully stabilizing our encoding format and will have a clear storage format documentation introduced in version 1.0. I agree that a more transparent format can provide users with a better sense of control, and we will try to create a human-readable format for exporting CRDT data (the kind that includes operation history). As for the application state, Loro already supports direct export in json format.

Re: Loro's rich text CRDT

#43

Post by fnordsensei »

injuly wrote: post #17
lewisjoe wrote: post #14

It's great work improvising over Peritext using joseph's latest CRDT work. Much needed literature in "applying CRDTs for richtext" space. But I'm surprised why this one too hasn't focussed a lot on rich-text block elements (like lists, tables & sections) as much as it focussed on text attributes (like bold and italics).

I've gone through the Peritext paper several times, and attempted to implement support for lists/tables/sections myself, but found it to be difficult owing to the limited scope of the original paper. I reached out to Martin Kleppman about this, and this is what he told me: "Thanks for your message. I've written up a document on how to extend Patreon with nested block elements such as bullet points. It's not properly…

I've recently implemented a rich text editor on top of Automerge and Peritext that supports blocks, inline-blocks (or whatever you call them: images, tables, etc.), unaware that this was in the works. The implementation ended up _very_ close to what's described in the linked post.

Re: Loro's rich text CRDT

#44

Post by rubymamis »

ghizmandier wrote: post #40
rubymamis wrote: post #2

Slightly off-topic - I don't think real-time collaboration is suitable for text-based formats. I believe collaboration similar to working with git is superior: 1. Fork the text 2. Submit proposal 3. Review 4. Merge/Cancel EDIT: To slightly expand on this - there are many reasons for this intuition - the main, IMO, is that people like to work on text privately before showing it to people. Also, the mental fear of your…

Hey, sounds like you'd love what we're building @ Ellipsus — https://ellipsus.com We started off your same exact assumptions and built a text editor that combines the best of both worlds: you can collaborate in real-time on the same piece; or branch off of it, work on your own and then have it reviewed and merged back into the main branch. That and other features that should make writing together a pleasure. Reach ou…

Looks awesome! Love the website. I signed up for the wait list.

Re: Loro's rich text CRDT

#45

Post by mixmastamyk »

jitl wrote: post #15
rubymamis wrote: post #13

Earlier quoted context omitted.

Git is just the underlying mechanism. What you show to the user depends on your UX/UI. You don't need to show users the words "fork/merge" even. How about: 1. "Edit Joe's text" 2. "Ask Joe for a review" 3. "Do you want to add Alice's changes?"

For one thing, Git is abysmal at merging rich text formats that need balancing open/close annotation markers, like HTML and similar. With line/character oriented merge algorithms syntax errors from merge are inevitable. Once you start pulling that thread, like “ok can we clean up such mistakes automatically with a better merge algorithm that understands the content a bit more?” you’ll end up back here at CRDTs. I’ve…

Re: markdown in vcs

Put each sentence and/or clause on its own line and it will avoid a large fraction of issues. Not all though.

Re: Loro's rich text CRDT

#46

Post by kiitos »

czx111331 wrote: post #41
kiitos wrote: post #29

Earlier quoted context omitted.

> if an operation has been synchronized across all endpoints, no new operations will occur concurrently with it, allowing it to be safely removed from the history. This assumes that the set of endpoints (really, nodes) is both well-known by all other nodes in the network, and stable over time (meaning new nodes will never be added). Even if this assumption can be made safely (which is not a given) the GC process desc…

Similar to OT, in certain scenarios, it's sufficient to ensure that only a subset of peers have the complete data, while others don't need the full history. For instance, in real-time collaboration scenarios with a central server, we can, just like OT, allow clients to hold only a shallow clone instead of the complete history. This approach results in minimal overhead for the clients.

I guess it all depends on how you define "client" and "shallow history", and the guarantees you provide around propagation of that history.

But if you have a central server that is considered to be the authoritative source of state, and assuming clients interact with that central server directly, then I'm not sure what is accomplished by modeling your data with CRDTs in the first place?

Re: Loro's rich text CRDT

#47

Post by mikebelanger »

czx111331 wrote: post #42
mikebelanger wrote: post #32

Looks neat! Would there be a way of intercepting state and making 'snapshots' into a more traditional format, like SQL, or even a JSON file? It sounds like this defaults to the server storing the whole state in their binary format, ditto the client-side portion of it. Nothing wrong with the format, but this is an early project, and nobody wants their data in something that's potentially unstable, or something that mi…

We are carefully stabilizing our encoding format and will have a clear storage format documentation introduced in version 1.0. I agree that a more transparent format can provide users with a better sense of control, and we will try to create a human-readable format for exporting CRDT data (the kind that includes operation history). As for the application state, Loro already supports direct export in json format.

Ok, I appreciate you reaching back. And my bad, I didn't see that it already supports direct export of JSON.

Re: Loro's rich text CRDT

#49

Post by lewisjoe »

josephg wrote: post #23
lewisjoe wrote: post #14

It's great work improvising over Peritext using joseph's latest CRDT work. Much needed literature in "applying CRDTs for richtext" space. But I'm surprised why this one too hasn't focussed a lot on rich-text block elements (like lists, tables & sections) as much as it focussed on text attributes (like bold and italics).

Adding to the other great responses - Coming from Google Wave, I still think of this problem the way we dealt with it in wave. In Wave, a document was a sequence of items + annotations. Items were either characters (collectively making up normal text), an item could be an embedded child item, like a table, image, nested document, etc. Embedded items were "in the document" just like any other document content, and cou…

By "annotation items" - you mean the special "boundary characters" that Loro CRDT uses to mark formatting boundaries right?

Yes, Google docs doesn't use such special annotation characters for bold/italics-like formattings. But it does use special boundary characters for marking "comment" boundaries. These characters simplify a few things from a comment perspective (as they shouldn't merge like formatting options). Zoho Writer uses a similar design for differentiating formatting boundaries vs comment boundaries.

> I'd like to take my own stab at rich text in diamond types soon, since I need it for a project. I think there's a cleaner way to do it - though until I write the code, who knows how it'll turn out.

Curious to take a look at your approach. Do write more on your blog Joseph :)

- Joe Lewis

Post reply on HN