Live data from Hacker News

Real Differences Between OT and CRDT for Co-Editors

arxiv.org

41–50 of 66 posts

Re: Real Differences Between OT and CRDT for Co-Editors

#41
post #23

Earlier quoted context omitted.

I've been working in OT systems for years (G Wave, ShareJS, ShareDB, some other stuff). I'm consistently surprised by how badly academic papers predict OT systems will perform. In reality, they perform great . My little C implementation of text OT can handle about 20M text operation transforms / second[1]. Part of the gap is that many academic papers model text operations as just single character edits. If you do tha…

I fully agree, I don't understand the emphasis on performance both in the literature and in comments on the subject. Additionally, it is more common to see CRDT libraries that make character-wise operations and create an object for each character (eg. y-js.org, github.com/google/ot-crdt-papers), which is not ideal. Obviously there are also libraries such as Atom's Teletype that have string-wise operations. The cynic…

[deleted]

Re: Real Differences Between OT and CRDT for Co-Editors

#42
post #23
post #16

Earlier quoted context omitted.

The time complexity of most OT systems is not related to H.

I've been working in OT systems for years (G Wave, ShareJS, ShareDB, some other stuff). I'm consistently surprised by how badly academic papers predict OT systems will perform. In reality, they perform great . My little C implementation of text OT can handle about 20M text operation transforms / second[1]. Part of the gap is that many academic papers model text operations as just single character edits. If you do tha…

My personal experience is different. OT tends to be quadratic in terms of computational complexity and codebase size. As long as you consider a simple case, 1^2 is 1. Then, add more data types (not just text), faulty servers, faulty network, non trivial network architectures... All these problems tend to interact. They easily go combinatorial, but again 1! is 1. I worked with an OT codebase that had C++ templates within C defines and I edited them using regexes (x types of operations => x^2 types of potential transformations).

OT aside, data sync in general is not a solved problem. Maybe I am especially unlucky, but I have yet to see a chat app that has no sync glitches. A chat is not the most difficult data structure to sync, but I saw bugs in Slack, Telegram, Instagram, Gchat, Skype, others... WhatsApp does not even try to sync...

A lot depends on the diversity of your use cases and the number of the nines you expect from the system...

Re: Real Differences Between OT and CRDT for Co-Editors

#43
post #14

Here's the key sentence: "concrete implementations of CRDT in co-editors revealed key missing steps in CRDT literature." This paper may be correct for academic CRDTs but it is very wrong when looking at industry implementations. My hunch is that because CRDTs are so much easier to grok than OT, engineers are empowered to make use case-specific improvements that aren't reflected in academic literature. For example, th…

Casual and blanket theoretical characterizations of OT and CRDT algorithms is exactly the kind of folly that the article is trying to address.

For example, it's a logical contradiction to begin the statement with "OT's complexity is _generally_ governed by editing history (H), i.e. O(H)" to conclude that "CRDTs have much better perf than OT", then adding a qualification that argument applies only for the specific off-line editing scenario (which btw is also not true).

It's also inaccurate to base CRDT's complexity on N - the document size, i.e. number of characters visible in the document. You need to include tombstones for the case of WOOT variants or use garbage collection for RGA (which then requires vector clocks). These nuances are described in sections 4.3.

OT's complexity is governed by O(c) where c is number of concurrent operations, period.

For off-line edits, the short answers is that OT's complexity is also not O(H), if H is the number of character edits, because you can easily apply compression. Now, the longer answer: there is pretty strong spatial locality in the distribution of edits over a document -- we don't sporadically and randomly add or delete characters around a document, but intuitively, most of the inserts will be adjacent characters (i.e. strings), and many of the deletes are over newly inserted strings. OT uses string-wise operations, hence it will compress n consecutive character insert ops down to a single string op. In addition you can compress delete edits over newly inserted content: i.e. [insert 'To be or not yoo be' at 0, delete 'y' at '14', delete 'o' at '14', insert 't' at '14'] => [insert 'To be or not to be']. This is basic idea behind "operation compression" which is what OT would use to support off-line, non-real-time, asynchronous editing, however you want to call it. There is a better example here (http://www3.ntu.edu.sg/home/czsun/projects/otfaq/#_Toc321146...).

Re: Real Differences Between OT and CRDT for Co-Editors

#44
post #10

An article of disappointing quality from well known OT authors. Like,algo x has issue X, algo y has issue Y, z has Z, so CRDT has issues X, Y and Z... and many things like that.

I co-authored the paper and stand by the rigor of the evidence and arguments. If you believe there is a mistake, I would be happy to have that discussion to the extent that substantive arguments and/or evidences are presented.

In referring to "CRDT", the paper qualifies upfront that it is "CRDT for co-editors" that we focus specifically on (title and 2nd sentence in the abstract). CRDT counters, sets etc are not in scope.

Re: Real Differences Between OT and CRDT for Co-Editors

#45
I enjoyed reading this paper. I also recently talked with Chris Colbert about his new plans to use a CRDT approach to collaborative editing of Jupyter notebooks. This made me curious again about how CoCalc’s collaborative editing is related to OT and CRDT approaches, so I wrote up a blog post about that just now. http://blog.sagemath.com/2018/10/11/collaborative-editing.ht...

Re: Real Differences Between OT and CRDT for Co-Editors

#46
post #23

Earlier quoted context omitted.

I've been working in OT systems for years (G Wave, ShareJS, ShareDB, some other stuff). I'm consistently surprised by how badly academic papers predict OT systems will perform. In reality, they perform great . My little C implementation of text OT can handle about 20M text operation transforms / second[1]. Part of the gap is that many academic papers model text operations as just single character edits. If you do tha…

I fully agree, I don't understand the emphasis on performance both in the literature and in comments on the subject. Additionally, it is more common to see CRDT libraries that make character-wise operations and create an object for each character (eg. y-js.org, github.com/google/ot-crdt-papers), which is not ideal. Obviously there are also libraries such as Atom's Teletype that have string-wise operations. The cynic…

I think you'll find the article supports your sentiment to an extent.

One of the key messages we tried to get across is that academic research for co-editing ought to go beyond theoretical analysis but rather drive research by pushing the envelope in new areas of application and/or features. OT's evolution has been symbiotic with new applications, while we haven't seen this in CRDT (for coediting) research. The reality is that you'll find new CRDT papers every year that still (a) make fairly broad claims of benefits are either theoretically dubious or not backed/validated by application/system implementations, (b) dwelling on technical issues that were resolve years ago. To move beyond this, we want to have put these issues in context (a general transformation approach) and dispel the most common fallacies in theoretical analysis.

Related to the 'visual-merging' feature you mentioned, when I worked on making MS Word collaborative as a research project, one of the UX research questions was how to allow users to visualize the different combined effects of updates to objects (e.g. if you changed the background of object X to red and I change it to yellow at the same time). We came up with a multi-versioning technique to display the potentially different combinations and help users select one that's the most desirable. It is definitely a interesting and challenging problem to consider for text documents.

Re: Real Differences Between OT and CRDT for Co-Editors

#47
post #44
post #10

An article of disappointing quality from well known OT authors. Like,algo x has issue X, algo y has issue Y, z has Z, so CRDT has issues X, Y and Z... and many things like that.

I co-authored the paper and stand by the rigor of the evidence and arguments. If you believe there is a mistake, I would be happy to have that discussion to the extent that substantive arguments and/or evidences are presented. In referring to "CRDT", the paper qualifies upfront that it is "CRDT for co-editors" that we focus specifically on (title and 2nd sentence in the abstract). CRDT counters, sets etc are not in s…

Well. On page 30 par 2 you conclude that OT has time complexity O(1) while CRDT is O(C) or so... I guess, you compare a rope-based OT implementation and a naive CRDT implementation (like, one doing a full rewrite on each key press). At least, that is my best guess.

Why don't you compare them the other way around? Rope-based CRDT and a naive OT? String splicing is O(N), you may put it on either side of the scales. Hashtable-hosted Causal Tree is O(1); hashtable-hosted RGA is basically the same.

I find the general discourse of the article not so useful. I may guess, you overreacted to certain marketing messages coming from the CRDT community.

Re: Real Differences Between OT and CRDT for Co-Editors

#48
post #27
post #11

Earlier quoted context omitted.

Yeah, I thought the whole point of OT was that it allowed any client to do resolution (because all operations transform other operations in a way to make them all resolve the same way no matter their order) Which is what made it seem like such overkill for Google Docs where there is a central authoritative server, so a much simpler architecture could have done the job...

Simple OT algorithms do work a lot better with a centralized server. With a centralized server, you can consider every resolution as happening between 2 nodes (server and client). This means transform / catchup is as simple as a for loop. In contrast, in decentralized context OT code needs to support Transform Property 2[1] in order to converge correctly in all cases. TP2 dramatically complicates the OT implementatio…

Very insightful, thank you for the detailed comment. About the DAG flattening algorithm you mention - in what way does this differ from a topological sorting? (https://en.wikipedia.org/wiki/Topological_sorting ) E.g. does it need to know about expensive or cheap combinations of operations, and try to avoid the former?

Re: Real Differences Between OT and CRDT for Co-Editors

#49
post #47
post #44

Earlier quoted context omitted.

I co-authored the paper and stand by the rigor of the evidence and arguments. If you believe there is a mistake, I would be happy to have that discussion to the extent that substantive arguments and/or evidences are presented. In referring to "CRDT", the paper qualifies upfront that it is "CRDT for co-editors" that we focus specifically on (title and 2nd sentence in the abstract). CRDT counters, sets etc are not in s…

Well. On page 30 par 2 you conclude that OT has time complexity O(1) while CRDT is O(C) or so... I guess, you compare a rope-based OT implementation and a naive CRDT implementation (like, one doing a full rewrite on each key press). At least, that is my best guess. Why don't you compare them the other way around? Rope-based CRDT and a naive OT? String splicing is O(N), you may put it on either side of the scales. Has…

Sorry, I can't follow your statements.

What is a rope-based CRDT?

What is the meaning of "String splicing is O(N)"

The String refers to what? And What is N?

Re: Real Differences Between OT and CRDT for Co-Editors

#50
post #28

If you're interested in this stuff, you might also like Raph Levien's writing on it: - https://medium.com/@raphlinus/towards-a-unified-theory-of-op... - https://github.com/xi-editor/xi-editor/blob/e8065a3993b80af0...

Thanks! I think a unification of OT and CRDT is a very promising space, both for practical implementation and for deeper academic understanding. To be honest, I only skimmed the posted paper, but it didn't grab my attention. It's very difficult these days for me to motivate spending time in this space.

TTF is a CRDT-like approach. They both maintain an extra data structure including tombstones. And TTF-operations are only applied on the designed data structure. To be applied on the document visible to users, TTF-operations need to be translated like CRDT.

Your solution is to combine GOTO with TTF. Actually, it is a combination of OT and CRDT.

Post reply on HN