Live data from Hacker News

OT and CRDT trade-offs for Real-Time collaboration

tiny.cloud

51–60 of 60 posts

Re: OT and CRDT trade-offs for Real-Time collaboration

#51
post #10

The only thing I remember from the CKEditor post from last year was that it took "42 man-years" to implement: https://news.ycombinator.com/item?id=18220020 I wonder if it's better understood now and can cost less R&D time.

It is, as I'll cover in the second post ;)

Re: OT and CRDT trade-offs for Real-Time collaboration

#52

Wanderlog ( https://wanderlog.com ) is a Google Docs for planning travel, and naturally, we had to figure this out early in the process. If you're on a Node.js/React stack, we highly recommend using the combination of OT-JSON0 [1] and ShareDB [2], two excellent libraries. OT-JSON0 lets you perform operational transforms on any JSON-serializable structure pretty intuitively, and ShareDB handles synchronizing it betwee…

atomic elements doesn't really extend to arbitrary HTML, which is fine for the simpler editing solutions but not TinyMCE.

To do something like that in arbitrary HTML you wind up locking at the top-level-block boundaries. Locking sounds great as an engineer but it leads to a garbage user experience.

Re: OT and CRDT trade-offs for Real-Time collaboration

#53

w00h00, this is my area of expertise. After 8 years of working on this, I have changed my thoughts: - The correct algorithm is not always the correct user experience. - End-to-end encryption is too important to not have. - Offline support is great, but it behaving consistently is more important than it behaving "intently". - Biggest pain points can most easily be solved at the editing layer, not data layer. As a resu…

User experience is definitely why we didn't go for an existing CRDT solution.

The model we've chosen is fairly agnostic to whether OT or CRDT is used to collaborate, as I'll expand on in the second post, so if we see an opportunity to switch to a CRDT-based solution we'll certainly jump on it!

You only need to normalise things like bold/italic/underline when using a browser DOM, every collaborative rich text model I've seen represents formatting as unordered attributes which is much easier to collaborate on.

Re: OT and CRDT trade-offs for Real-Time collaboration

#54
post #32

Earlier quoted context omitted.

Preserving the position of the cursor is a crucial feature for collab editors. However, he doesnt go into detail explaining why CRDTs make this feature impossible. Sure the CRDT route splits the text, but I fail to see why that makes cursor pos preservation impossible?

It's not impossible, particularly in cases where the model is fairly flat. In complex models it is symptomatic of a larger issue with complexity in CRDTs; the post was already quite long and I didn't want to repeat any more of the research I'd linked to.

You mention that example as a turning point for choosing OT, confirming your research. But to an outsider it is not particularly convincing. Especially since you’d expect a CRDT implementation to use the rope/tombstone structure, applying the transform to that sequence of characters and not actually have to split a text node, which looks like a limitation of this particular implementation, or tight coupling between the data model and the DOM.

The post is a good introduction to OT and CRDT, but is very light on actual trade-offs and seems to end with a gut-feeling conclusion, when the title promised “trade-offs between”.

Re: OT and CRDT trade-offs for Real-Time collaboration

#55

Earlier quoted context omitted.

JSON.stringify('a') https://en.wikipedia.org/wiki/Lexicographical_order

Cool! Make sense. Let's say our state have an array like ['a','b']. What about if Peer-A splice the first element (0) which results in ['b'] and Peer-B wants to mutate 'a' to 'A' which results in ['A','b'] and they both make the mutation at the same millisecond?

You have to choose which type of Array:

1. An atomic Array

2. Fixed index Array

3. Linked list

Now rather than not knowing what magic is going to happen, app developers know what to expect in advance.

So atomic would choose one or the other, not a combined result.

Fixed array 2nd item to 1st which nulls 2nd and makes 1st b, simultaneously also makes 1st A. Cause lower case b is lexically larger than A, you'd get [b, null].

Linked list theoretically could handle it as you suggest.

Re: OT and CRDT trade-offs for Real-Time collaboration

#56

Earlier quoted context omitted.

Cool! Make sense. Let's say our state have an array like ['a','b']. What about if Peer-A splice the first element (0) which results in ['b'] and Peer-B wants to mutate 'a' to 'A' which results in ['A','b'] and they both make the mutation at the same millisecond?

You have to choose which type of Array: 1. An atomic Array 2. Fixed index Array 3. Linked list Now rather than not knowing what magic is going to happen, app developers know what to expect in advance. So atomic would choose one or the other, not a combined result. Fixed array 2nd item to 1st which nulls 2nd and makes 1st b, simultaneously also makes 1st A. Cause lower case b is lexically larger than A, you'd get [b,…

Does gun have a built-in "Linked list" in the API? If so, where are the docs? Could not find them.

If not, any paper or doc that explain how to implement it with gun?

Re: OT and CRDT trade-offs for Real-Time collaboration

#57

Earlier quoted context omitted.

You have to choose which type of Array: 1. An atomic Array 2. Fixed index Array 3. Linked list Now rather than not knowing what magic is going to happen, app developers know what to expect in advance. So atomic would choose one or the other, not a combined result. Fixed array 2nd item to 1st which nulls 2nd and makes 1st b, simultaneously also makes 1st A. Cause lower case b is lexically larger than A, you'd get [b,…

Does gun have a built-in "Linked list" in the API? If so, where are the docs? Could not find them. If not, any paper or doc that explain how to implement it with gun?

Started, but never finished. :(

@nmaro has some pretty good stuff.

The cartoon explainers gives an overview.

But you'd probably want to do RGA algorithm or one of the newer ones.

Martin Kleppmann has a pretty good talk with bunch of paper references:

https://youtu.be/yCcWpzY8dIA

Re: OT and CRDT trade-offs for Real-Time collaboration

#58
post #54

Earlier quoted context omitted.

It's not impossible, particularly in cases where the model is fairly flat. In complex models it is symptomatic of a larger issue with complexity in CRDTs; the post was already quite long and I didn't want to repeat any more of the research I'd linked to.

You mention that example as a turning point for choosing OT, confirming your research. But to an outsider it is not particularly convincing. Especially since you’d expect a CRDT implementation to use the rope/tombstone structure, applying the transform to that sequence of characters and not actually have to split a text node, which looks like a limitation of this particular implementation, or tight coupling between t…

It's hard to offer more detail than that in a blog post. I'm not going to repeat the research papers or write one of my own. The post is already so long and complex, covering so many surface details, I couldn't reasonably push it any further.

Re: OT and CRDT trade-offs for Real-Time collaboration

#59
post #54

Earlier quoted context omitted.

You mention that example as a turning point for choosing OT, confirming your research. But to an outsider it is not particularly convincing. Especially since you’d expect a CRDT implementation to use the rope/tombstone structure, applying the transform to that sequence of characters and not actually have to split a text node, which looks like a limitation of this particular implementation, or tight coupling between t…

It's hard to offer more detail than that in a blog post. I'm not going to repeat the research papers or write one of my own. The post is already so long and complex, covering so many surface details, I couldn't reasonably push it any further.

I’d suggest skipping right to the meat of the subject (pros/cons in this case), assuming some previous knowledge, if that’s what the article proposes to share, and simply point readers to learning resources; it’s not possible to start from scratch on every topic. The deeper the subject, the more glaring this problem becomes.

Re: OT and CRDT trade-offs for Real-Time collaboration

#60

Earlier quoted context omitted.

Does gun have a built-in "Linked list" in the API? If so, where are the docs? Could not find them. If not, any paper or doc that explain how to implement it with gun?

Started, but never finished. :( @nmaro has some pretty good stuff. The cartoon explainers gives an overview. But you'd probably want to do RGA algorithm or one of the newer ones. Martin Kleppmann has a pretty good talk with bunch of paper references: https://youtu.be/yCcWpzY8dIA

Oh, I see. Just wondering, How GUN handle a situation like described above when working with Arrays?
Post reply on HN