Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Loro's rich text CRDT

loro.dev

Post reply on HN
Print view View original

21–30 of 49 posts

Re: Loro's rich text CRDT

#21

Post by hwillis »

lewisjoe wrote: post #14

It's great work improvising over Peritext using joseph's latest CRDT work. Much needed literature in "applying CRDTs for richtext" space. But I'm surprised why this one too hasn't focussed a lot on rich-text block elements (like lists, tables & sections) as much as it focussed on text attributes (like bold and italics).

> But I'm surprised why this one too hasn't focussed a lot on rich-text block elements (like lists, tables & sections)

This is where the real strength of crdts is, IMO. Tree-like structures turn into DAGs once you have multiple edits happening (two users editing a tree node create children with two parents) and are much more interesting than linear data like a string. It's definitely not the most efficient way to store data, but it's incredibly convenient for reconciling versions of pretty complex UI.

A few years ago I threw together a report-writing app for car crash data that let you drop different graphs, maps, visualizations, table excerpts (eg "select top 5 worst roads") that could all reference each other. You make a report for your hometown, and some other team could fork it to see the results for their area.

Data were all update live, even if it was pinned to a specific time, since things could be updated after the fact. That other team could then add in a new section of the report or change the conclusion section, and still get the first team's updates to the shared sections (with an option to revert).

You can do like... most stuff like that, if you want. Slides, tables, drawings, whatever. It's not great for graphs but idk what is good for graphs.

Re: Loro's rich text CRDT

#22

Post by rubymamis »

bmacho wrote: post #18
josephg wrote: post #12

Earlier quoted context omitted.

Yeah; and the merging is correct in all cases. You don't get spurious conflicts like you can with git. Also the same system can work both in realtime and offline scenarios. And CRDTs can handle a lot more than just plain text editing. That said, one thing git does that I like is that sometimes I want conflict markers to be added to my document when concurrent edits happen on the same line. CRDTs (and REGs like this)…

> Yeah; and the merging is correct in all cases. No, it's obviously not. I'd say most of the time the merging is incorrect, that is, it is in contrast to both of the parties intention. Unlike git, which actually notifies them, Loro just silently modifies both changes into something nonsense/wrong.

Exactly. This process of putting users in control by notifying them about changes and allowing them to decide how to merge is essential when dealing with text.

It sounds like Loro can do that too, but then again, is it worth the complexity over git?

Re: Loro's rich text CRDT

#23

Post by josephg »

lewisjoe wrote: post #14

It's great work improvising over Peritext using joseph's latest CRDT work. Much needed literature in "applying CRDTs for richtext" space. But I'm surprised why this one too hasn't focussed a lot on rich-text block elements (like lists, tables & sections) as much as it focussed on text attributes (like bold and italics).

Adding to the other great responses -

Coming from Google Wave, I still think of this problem the way we dealt with it in wave. In Wave, a document was a sequence of items + annotations. Items were either characters (collectively making up normal text), an item could be an embedded child item, like a table, image, nested document, etc. Embedded items were "in the document" just like any other document content, and could be inserted or deleted the same as text.

Then annotations were used for formatting, like bolded regions or links. This is how peritext and quill work. There are no "annotation items" (since managing them sounds tedious).

It looks like loro works slightly differently. I'd like to take my own stab at rich text in diamond types soon, since I need it for a project. I think there's a cleaner way to do it - though until I write the code, who knows how it'll turn out.

As others have said, once nice thing about this model is that embedded items can themselves be targets of editing events. For example, you could add a map to google wave and then users could collaboratively add and remove pins to the map.

Re: Loro's rich text CRDT

#24

Post by josephg »

bmacho wrote: post #18
josephg wrote: post #12

Earlier quoted context omitted.

Yeah; and the merging is correct in all cases. You don't get spurious conflicts like you can with git. Also the same system can work both in realtime and offline scenarios. And CRDTs can handle a lot more than just plain text editing. That said, one thing git does that I like is that sometimes I want conflict markers to be added to my document when concurrent edits happen on the same line. CRDTs (and REGs like this)…

> Yeah; and the merging is correct in all cases. No, it's obviously not. I'd say most of the time the merging is incorrect, that is, it is in contrast to both of the parties intention. Unlike git, which actually notifies them, Loro just silently modifies both changes into something nonsense/wrong.

My correctness point is that Git can end up with commits which spuriously conflict with themselves. Git's merge algorithms simply aren't very good.

CRDTs (and REG) are correct in that everyone always ends up seeing the same document state. But I take your point - which is essentially that Loro (and automerge, yjs, diamond types, etc) don't correctly preserve intent in all cases. As you point out, if two users concurrently edit the same word, you can get nonsense.

In practice thats usually fine when users edit in realtime - since users notice and fix it. But its often not ok while users edit asyncronously (eg while offline). Thats exactly why I want a CRDT type system which can emit git style conflicts.

Re: Loro's rich text CRDT

#25

Post by josephg »

jitl wrote: post #15
rubymamis wrote: post #13

Earlier quoted context omitted.

Git is just the underlying mechanism. What you show to the user depends on your UX/UI. You don't need to show users the words "fork/merge" even. How about: 1. "Edit Joe's text" 2. "Ask Joe for a review" 3. "Do you want to add Alice's changes?"

For one thing, Git is abysmal at merging rich text formats that need balancing open/close annotation markers, like HTML and similar. With line/character oriented merge algorithms syntax errors from merge are inevitable. Once you start pulling that thread, like “ok can we clean up such mistakes automatically with a better merge algorithm that understands the content a bit more?” you’ll end up back here at CRDTs. I’ve…

Yep. And git can't at all handle realtime editing (since you'd need to commit & push after every keystroke). Or handle any non-text data. Even markdown, as you mentioned, can be a pain.

CRDTs should be able to solve all of these problems. One thing we're still missing in CRDTs is git style conflicts when merging long running branches. Should be possible - but still nobody has solved that as far as I know.

Re: Loro's rich text CRDT

#27

Post by samatman »

wim wrote: post #16
rubymamis wrote: post #7

Earlier quoted context omitted.

So they mention my exact concerns and addressed them, very cool. But their solution in practice doesn't look very different from what we can already achieve with git (apart from seeing your collaborator changes in real-time, which I'm not sure how substantial it is), or am I missing something? At the end of the day, how will this look to the end user? > a user may work in isolation on their own copy of a document for…

Another issue which git won't solve is if you collaborate on text which has "structure". For example when merging indented (todo) lists as flat text, bullets might end up in the wrong place, e.g. [ ] Project [ ] Don't do this [ ] Task A While offline, user #1 adds '[ ] Task B' and '[ ] rm -rf /' under "Don't Do This", while user #2 adds '[ ] Do This' under 'Project'. Obviously you don't want to merge this as: [ ] Pro…

I believe that handling merges like this correctly was a motive for designing pijul: https://pijul.org

See the item on the splash page about 'merge correctness'. Unfortunately I wasn't able to find the post detailing the behavior with a bit of searching.

Re: Loro's rich text CRDT

#28

Post by layer8 »

madeofpalk wrote: post #3
rubymamis wrote: post #2

Slightly off-topic - I don't think real-time collaboration is suitable for text-based formats. I believe collaboration similar to working with git is superior: 1. Fork the text 2. Submit proposal 3. Review 4. Merge/Cancel EDIT: To slightly expand on this - there are many reasons for this intuition - the main, IMO, is that people like to work on text privately before showing it to people. Also, the mental fear of your…

This might be true for writing an essay. But plenty of collaborative text-editing happens on a much smaller/informal/ad hoc scale. My personal experience of using Google Docs/Sheets strongly disagrees with you.

Asynchronous editing on a centralized platform only infrequently causes actual conflicts in practice, like multiple users editing the same sentence at the same time. Either the editing is actually sequential in time, or independent parts of the document are being edited. With asynchronous decentralized/offline editing, conflicts become more likely.

Re: Loro's rich text CRDT

#29

Post by kiitos »

jitl wrote: post #6

I'm curious about this line describing REG: > The REG algorithm excels with its fast local update speeds and eliminate concerns about tombstone collection in CRDTs. For instance, if an operation has been synchronized across all endpoints, no new operations will occur concurrently with it, allowing it to be safely removed from the history. If you remove these ops from history, does that remove the ability to time trav…

> if an operation has been synchronized across all endpoints, no new operations will occur concurrently with it, allowing it to be safely removed from the history.

This assumes that the set of endpoints (really, nodes) is both well-known by all other nodes in the network, and stable over time (meaning new nodes will never be added).

Even if this assumption can be made safely (which is not a given) the GC process described here is still an optimization, which would be subverted when even a single node in the network became slow or broken.

It's also basically orthogonal to the concept of "tombstones", which are still required if you want to delete anything from the data structure.

Post reply on HN