Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OT and CRDT trade-offs for Real-Time collaboration

tiny.cloud

Post reply on HN
Print view View original

31–40 of 60 posts

Re: OT and CRDT trade-offs for Real-Time collaboration

#31

Post by phedkvist »

Thanks for linking to my blog post :)

"This is the exact "split node" scenario I described earlier; applying bold near the start of a text node necessarily has to split the text node into three parts (before, bold, and after)."

Not sure why you split it up into sections. In my CRDT implementation, I would add meta-data to each character, with the boolean property which is either bold or not. It's certainly cumbersome to keep the cursor at the right place when inserts are being made, but it's doable. https://pierrehedkvist.com/posts/1-creating-a-collaborative-...

I personally never understood how OT actually works, clearly, Google Docs and others find it useful. But to me, CRDT has more solid proof and reasoning behind it, and it is easier to comprehend. https://medium.com/@pierrehedkvist/creating-a-collaborative-...

Re: OT and CRDT trade-offs for Real-Time collaboration

#32

Post by eudoxus »

Vinnl wrote: post #30
jiofih wrote: post #29

Earlier quoted context omitted.

Because it doesn’t highlight a particularly relevant deficiency of the CRDT protocol. Preserving cursor position is not hard in that case. The post even mentions stronger downsides of OT, the example does not support the conclusion. Why is OT better?

Huh, the article gave me the impression that preserving the position was a crucial problem there. As for the last question, I think the later posts will get to that.

Preserving the position of the cursor is a crucial feature for collab editors. However, he doesnt go into detail explaining why CRDTs make this feature impossible. Sure the CRDT route splits the text, but I fail to see why that makes cursor pos preservation impossible?

Re: OT and CRDT trade-offs for Real-Time collaboration

#33

Post by eudoxus »

ergl wrote: post #28
codebeaker wrote: post #23

Earlier quoted context omitted.

I have been using Automerge recently for a project, and I have found it to be very, very user-friendly. Our use-case is offline-editing of documents with an eventual sync-with-yourself-online. It's mostly there as a sync tool, not as a p2p colalb editing. Unless you count yourself as a peer, I guess! We store a document in local storage which is the result of `Automerge.save(automergedoc) => serializable string`. Tha…

Yeah, personally the feature I'm looking forward to is materializing "changes" into a document without losing editing history. At some point your set of changes grows too large or you want to trim it to, say, a few weeks or months. Afaik that isn't possible yet without losing the ability to merging later on.

I'm farmilliar with Automerge, but not its specific features on state compression. In general change set compression is possible with CRDTs but there are some "gotchas" you need to look out for. Most notably, all peers interested in a changeset, have to be completely in sync at the moment of compression, otherwise, depending on which underlying CRDT type you use, there can end up with duplicate entries in for example a Set type. Depending on the use case, 100% peer sync is possible, in others its more challenging.

Re: OT and CRDT trade-offs for Real-Time collaboration

#34

Post by migueloller »

phsource wrote: post #12

Wanderlog ( https://wanderlog.com ) is a Google Docs for planning travel, and naturally, we had to figure this out early in the process. If you're on a Node.js/React stack, we highly recommend using the combination of OT-JSON0 [1] and ShareDB [2], two excellent libraries. OT-JSON0 lets you perform operational transforms on any JSON-serializable structure pretty intuitively, and ShareDB handles synchronizing it betwee…

We’re actually using ShareDB at Makeswift right now and for us it has become a horrible burden. There are very subtle bugs on their data persistence layer with concurrency and the API is a bit of a mess to work with in my opinion. I’m glad to see other people are having success with it, though.

Re: OT and CRDT trade-offs for Real-Time collaboration

#35

Post by migueloller »

sipjca wrote: post #14
migueloller wrote: post #3

I’ve been doing a pretty deep dive on CRDTs and OTs to get the right user experience for our collaborative website builder[1]. I’m still not done with the implementation and subsequent testing (i.e., splitting text nodes, as mentioned in the post). But the core algorithm behind Automerge[2], formalized in the OpSets paper[3] is extremely promising. A good example of its power is the ability to do an atomic tree “move…

I think that blog post would be a great idea, would love to read it

Really? I’ll get something out this week. My Twitter handle is in my profile. I’ll probably tweet about it when I write it.

Re: OT and CRDT trade-offs for Real-Time collaboration

#36

Post by marknadal »

mkl wrote: post #25
marknadal wrote: post #16

w00h00, this is my area of expertise. After 8 years of working on this, I have changed my thoughts: - The correct algorithm is not always the correct user experience. - End-to-end encryption is too important to not have. - Offline support is great, but it behaving consistently is more important than it behaving "intently". - Biggest pain points can most easily be solved at the editing layer, not data layer. As a resu…

With end-to-end encryption in GUN, what role does the server play? I assume the merging type activity happens in the clients? BTW, the name makes it almost impossible to search for info about GUN.

Yes, merging is per peer, not server transformed.

"Servers" act as WebRTC bootstrapping signals, storage backups, relay/routing, etc., and you can have as many of them as you want (no need to be centralized).

True. `gunDB` tag should help a little.

Re: OT and CRDT trade-offs for Real-Time collaboration

#37

Post by marknadal »

josemaenzo wrote: post #27
marknadal wrote: post #16

w00h00, this is my area of expertise. After 8 years of working on this, I have changed my thoughts: - The correct algorithm is not always the correct user experience. - End-to-end encryption is too important to not have. - Offline support is great, but it behaving consistently is more important than it behaving "intently". - Biggest pain points can most easily be solved at the editing layer, not data layer. As a resu…

Do you have any document that explain what resolution algorithm uses in what cases? For example, one peer change a property value and the other peer deletes it.

Same algorithm for everything.

This cartoon explains it:

https://gun.eco/distributed/matters.html

It is a vector + timestamp + lexical sort.

A delete is changing a value to `null`, it would lose (I assume you are asking if/when these 2 changes happen at the exact same microsecond time, conflicting?) as is it has a lower lexical rank.

Re: OT and CRDT trade-offs for Real-Time collaboration

#38

Post by phsource »

migueloller wrote: post #34
phsource wrote: post #12

Wanderlog ( https://wanderlog.com ) is a Google Docs for planning travel, and naturally, we had to figure this out early in the process. If you're on a Node.js/React stack, we highly recommend using the combination of OT-JSON0 [1] and ShareDB [2], two excellent libraries. OT-JSON0 lets you perform operational transforms on any JSON-serializable structure pretty intuitively, and ShareDB handles synchronizing it betwee…

We’re actually using ShareDB at Makeswift right now and for us it has become a horrible burden. There are very subtle bugs on their data persistence layer with concurrency and the API is a bit of a mess to work with in my opinion. I’m glad to see other people are having success with it, though.

Ah that's unfortunate; we did find it a bit weird that you have to think of all changes as `ops`, but otherwise, I'm curious as to what you've had issues with.

We also have a custom back-end that's MySQL that may have changed how it work a bit

Re: OT and CRDT trade-offs for Real-Time collaboration

#39

Post by aloer »

alangibson wrote: post #21
phsource wrote: post #12

Wanderlog ( https://wanderlog.com ) is a Google Docs for planning travel, and naturally, we had to figure this out early in the process. If you're on a Node.js/React stack, we highly recommend using the combination of OT-JSON0 [1] and ShareDB [2], two excellent libraries. OT-JSON0 lets you perform operational transforms on any JSON-serializable structure pretty intuitively, and ShareDB handles synchronizing it betwee…

I came to the same "LWW on fragments" solution for peraspera.io. In a lot of cases, all OT gets you is being able to watch someone else's cursor move around. The new wore off on that a long time ago.

Could you elaborate a bit on that?

Re: OT and CRDT trade-offs for Real-Time collaboration

#40

Post by marknadal »

throwaway739103 wrote: post #24
marknadal wrote: post #16

w00h00, this is my area of expertise. After 8 years of working on this, I have changed my thoughts: - The correct algorithm is not always the correct user experience. - End-to-end encryption is too important to not have. - Offline support is great, but it behaving consistently is more important than it behaving "intently". - Biggest pain points can most easily be solved at the editing layer, not data layer. As a resu…

You are a charming and energized speaker and perhaps that is why Tim Draper thinks you a good investment as with others he has made. Acronyms like 'PTSD' and 'PARTY' in the project is likely good at attracting people who think they sound fun and do not question many details. It is difficult to get brief clear details about how gun.js works. There are distracting or simple documents, discussions about high level ideas…

Thank you!

They also like the math & science behind it, some investors do DD, some don't.

You're tone is pretty assuming, smug, and insulting.

It is running in production with millions of people, yes there are some hiccups, but surviving & we're improving it to scale even more.

I rebuild GUN from scratch in 30min on stage, that is how little/simple GUN is:

https://gun.eco/docs/Porting-GUN

Do I pay myself? Lol. Yes. No, I do not pay OSS, but yes people contribute regardless, strangers help me, my kids help me, investors help me, family & friends help me.

Post reply on HN