Live data from Hacker News

A Conflict-Free Replicated JSON Datatype

arxiv.org

1–10 of 60 posts

Re: A Conflict-Free Replicated JSON Datatype

#4
> Our principle of not losing input due to concurrent modifications appears reasonable, but as illustrated in Figure 4, it leads to merged document states that may be surprising to application programmers who are more familiar with sequential programs.

That decision is a bit odd to me. Not only do they show that their merge system can produce data that the application considers invalid, it can even convert a string to a list without having any operation creating a list explicitly.

> Moreover, garbage collection (tombstone removal) is required in order to prevent unbounded growth of the datastructure

That is always the painful part of a CRDT system. Until that part is done, it cannot really be used in production. That said, previous work on CRDTs give me confidence that it can be implemented with minimal overhead.

> In fact, we wrote the LATEX source text of this paper using an experimental collaborative text editor that is based on our implementation of this CRDT. We are making the character-by-character editing trace of this document available as supplemental data of this paper

String editing is brushed over in the paper. I'd love to see their implementation. Does anyone know where it all is?

Re: A Conflict-Free Replicated JSON Datatype

#5

> Our principle of not losing input due to concurrent modifications appears reasonable, but as illustrated in Figure 4, it leads to merged document states that may be surprising to application programmers who are more familiar with sequential programs . That decision is a bit odd to me. Not only do they show that their merge system can produce data that the application considers invalid, it can even convert a string…

> That decision is a bit odd to me. Not only do they show that their merge system can produce data that the application considers invalid, it can even convert a string to a list without having any operation creating a list explicitly.

It's hard to see good alternatives there. If you move from a single editor to concurrent edits without very strict ordering guarantees or a transactional system, you will need to deal with conflicts or lose data. Creating output that would be considered invalid unless the application is fixed to explicitly deal with it (or explicitly throw it away) seems quite reasonable.

Re: A Conflict-Free Replicated JSON Datatype

#6

> Our principle of not losing input due to concurrent modifications appears reasonable, but as illustrated in Figure 4, it leads to merged document states that may be surprising to application programmers who are more familiar with sequential programs . That decision is a bit odd to me. Not only do they show that their merge system can produce data that the application considers invalid, it can even convert a string…

> String editing is brushed over in the paper.

LaTeX source code is basically just a string. It would have been more interesting to see this applied to a hierarchical data-structure, such as HTML, where e.g. elements are nested inside markup nodes.

Re: A Conflict-Free Replicated JSON Datatype

#8
It is impossible to have accurate conflict-free replicated data. Unless the users are given the opportunity to manually resolve conflicts, it will never be 100% accurate - Incorrect data will creep in from time to time (but at least the incorrect data will be consistent across all nodes).

The root of the problem is that the system cannot understand the collaborative intent of concurrent users - If you have 2 users who made a change at the same time (without being aware of each other); you have to account for the fact that maybe UserB would have behaved differently if they had been aware of UserA's input (which happened at the same time while one or both users were offline). If the user has been offline for a while - Many such conflicts could arise (maybe by the time the internet comes back on, the user is looking at a completely different page than the one they made the change on) and it's tedious to make the user resolve them all manually.

Also with this approach, it tends to force you to keep a copy/cache of all the data in your entire app (for that logged-in user) on the frontend - If you have a big app with lots of pages, that could consume a lot of memory.

There are cases where the best solution is to simply tell the user "Sorry, you do not have an internet connection at the moment, so you cannot modify this data" rather than giving them a false sense that the data are correctly backed up in the cloud. I think with CRDTs, it's really important to inform the user when they are offline and when their data are not synced/backed up in the cloud (so they don't get any bad surprises when the internet suddenly comes back on).

CRDTs are good where the accuracy of the data is not critical (E.g. bank transactions). One could argue that they improve the user experience, but at the core, developers like them because they make life easier.

Re: A Conflict-Free Replicated JSON Datatype

#9
post #5

> Our principle of not losing input due to concurrent modifications appears reasonable, but as illustrated in Figure 4, it leads to merged document states that may be surprising to application programmers who are more familiar with sequential programs . That decision is a bit odd to me. Not only do they show that their merge system can produce data that the application considers invalid, it can even convert a string…

> That decision is a bit odd to me. Not only do they show that their merge system can produce data that the application considers invalid, it can even convert a string to a list without having any operation creating a list explicitly. It's hard to see good alternatives there. If you move from a single editor to concurrent edits without very strict ordering guarantees or a transactional system, you will need to deal w…

For this you would want a type / schema that explicitly represented the conflicts at the right level of granularity and helped applications resolve them.

Whereas producing an object the pretends to be conflict free, but has some fields that might or might not change type is the worst of both worlds.

Re: A Conflict-Free Replicated JSON Datatype

#10

Related: does anyone know any other good resources for architecting and building OT/CRDT for nested data structures like HTML or JSON? Ideally projects with working examples even?

I'm looking for a good 3-way merge for strings implemented in Javascript.
Post reply on HN